[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:43:48 dist-test-slave-njxd krb5kdc[20862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:43:48 dist-test-slave-njxd krb5kdc[20862](info): set up 2 sockets
Aug 10 11:43:48 dist-test-slave-njxd krb5kdc[20862](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 11:43:50.019423 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:43:50 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:43:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:43:50.234611 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:36647
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:50.587029 20877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:50.587544 20877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:50.587778 20877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:50.595726 20877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:43:50.595856 20877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:50.595942 20877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:50.596045 20877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:43:50.596132 20877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:43:50.607003 20877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:36647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:36647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:50.608505 20877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:50.610111 20877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:50.626293 20883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:50.626314 20884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:50.626942 20886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:50.627242 20877 server_base.cc:1034] running on GCE node
I20260810 11:43:50.628328 20877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:50.630029 20877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:50.631309 20877 hybrid_clock.cc:648] HybridClock initialized: now 1786362230631269 us; error 39 us; skew 500 ppm
I20260810 11:43:50.631723 20877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:50 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362230, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:50.635432 20877 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:43:50.637096 20877 webserver.cc:460] Webserver started at http://127.20.93.190:38443/ using document root <none> and password file <none>
I20260810 11:43:50.637964 20877 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:50.638126 20877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:50.638468 20877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:50.642164 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "5c4e552911be4446ada130053ea8c941"
format_stamp: "Formatted at 2026-08-10 11:43:50 on dist-test-slave-njxd"
server_key: "25eea6f11d91cfcb081f8c03d5f635ea"
server_key_iv: "5e619f215f0793eeb607c3ce96f00e4e"
server_key_version: "encryptionkey@0"
I20260810 11:43:50.643074 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "5c4e552911be4446ada130053ea8c941"
format_stamp: "Formatted at 2026-08-10 11:43:50 on dist-test-slave-njxd"
server_key: "25eea6f11d91cfcb081f8c03d5f635ea"
server_key_iv: "5e619f215f0793eeb607c3ce96f00e4e"
server_key_version: "encryptionkey@0"
I20260810 11:43:50.649199 20877 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:43:50.653429 20895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:50.655102 20877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 11:43:50.655319 20877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "5c4e552911be4446ada130053ea8c941"
format_stamp: "Formatted at 2026-08-10 11:43:50 on dist-test-slave-njxd"
server_key: "25eea6f11d91cfcb081f8c03d5f635ea"
server_key_iv: "5e619f215f0793eeb607c3ce96f00e4e"
server_key_version: "encryptionkey@0"
I20260810 11:43:50.655550 20877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:50.671265 20877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:50.676066 20877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:50.676388 20877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:50.697216 20877 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:36647
I20260810 11:43:50.697257 20952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:36647 every 8 connection(s)
I20260810 11:43:50.698837 20877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:43:50.701324 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 20877
I20260810 11:43:50.701723 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:43:50.703202 20953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:43:50.703775 20854 external_mini_cluster.cc:1428] Setting key 0fc48cdb37bbe5e12235a629ffdc1fc0
I20260810 11:43:50.715736 20953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941: Bootstrap starting.
I20260810 11:43:50.720309 20953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:50.721979 20953 log.cc:824] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:50.725754 20953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941: No bootstrap required, opened a new log
I20260810 11:43:50.737717 20953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } }
I20260810 11:43:50.738348 20953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:50.738528 20953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4e552911be4446ada130053ea8c941, State: Initialized, Role: FOLLOWER
I20260810 11:43:50.739355 20953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } }
I20260810 11:43:50.739704 20953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:43:50.739866 20953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:43:50.740079 20953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:50.743201 20953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } }
I20260810 11:43:50.743881 20953 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4e552911be4446ada130053ea8c941; no voters:
I20260810 11:43:50.744659 20953 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:43:50.744817 20958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:43:50.746186 20958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4e552911be4446ada130053ea8c941, State: Running, Role: LEADER
I20260810 11:43:50.746801 20958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } }
I20260810 11:43:50.747408 20953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 11:43:50 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:50.749655 20960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4e552911be4446ada130053ea8c941. Latest consensus state: current_term: 1 leader_uuid: "5c4e552911be4446ada130053ea8c941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } } }
I20260810 11:43:50.750074 20960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:50.750670 20959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c4e552911be4446ada130053ea8c941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4e552911be4446ada130053ea8c941" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36647 } } }
I20260810 11:43:50.751029 20959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941 [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:50.755149 20963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:43:50.764118 20963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:43:50.767254 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:50.708056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.20.93.190:36647)
0810 11:43:50.709060 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:50.709085 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:50.717892 (+ 8807us) server_negotiation.cc:407] Connection header received
0810 11:43:50.731760 (+ 13868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:50.731788 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:50.732294 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:50.732687 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:50.737131 (+ 4444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:50.738809 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:50.741675 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:50.742475 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:50.742634 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:50.751933 (+ 9299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:50.751952 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:50.751985 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:50.752064 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:50.757423 (+ 5359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:50.758931 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:50.758944 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:50.758956 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:50.759305 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:50.760140 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:50.760148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:50.760155 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:50.761381 (+ 1226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:50.764890 (+ 3509us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:50.765207 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 11:43:50.765698 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":415,"threads_started":1}
I20260810 11:43:50.776364 20963 catalog_manager.cc:1347] Generated new cluster ID: de005d2827cd428c996ef2ba2ed7e2ce
I20260810 11:43:50.776497 20963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:43:50.791813 20963 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:43:50.793826 20963 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:43:50.801981 20963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4e552911be4446ada130053ea8c941: Generated new TSK 0
I20260810 11:43:50.802949 20963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:43:50.862500 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:51.221982 20985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:51.222313 20985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:51.222486 20985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:51.229900 20985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:51.230047 20985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:51.230235 20985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:43:51.240141 20985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:51.241459 20985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:51.242868 20985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:51.256058 20990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:51.256206 20993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:51.256327 20991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:51.257267 20985 server_base.cc:1034] running on GCE node
I20260810 11:43:51.257923 20985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:51.258862 20985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:51.260077 20985 hybrid_clock.cc:648] HybridClock initialized: now 1786362231260009 us; error 48 us; skew 500 ppm
I20260810 11:43:51.260411 20985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:51 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362231, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:51.263655 20985 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:43:51.265084 20985 webserver.cc:460] Webserver started at http://127.20.93.129:38285/ using document root <none> and password file <none>
I20260810 11:43:51.265867 20985 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:51.266007 20985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:51.266332 20985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:51.269217 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "5056e332357645d79cea21a07ef1194e"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "0f6ec9999c243df2282473341ae351c9"
server_key_iv: "7a4b2485f9ae2581a412eb2a208c3467"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.270010 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "5056e332357645d79cea21a07ef1194e"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "0f6ec9999c243df2282473341ae351c9"
server_key_iv: "7a4b2485f9ae2581a412eb2a208c3467"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.275202 20985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:43:51.278788 21002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.280196 20985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:43:51.280388 20985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "5056e332357645d79cea21a07ef1194e"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "0f6ec9999c243df2282473341ae351c9"
server_key_iv: "7a4b2485f9ae2581a412eb2a208c3467"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.280594 20985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:51.311120 20985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:51.315541 20985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:51.315862 20985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:51.316975 20985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:51.318442 20985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:51.318552 20985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.318702 20985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:51.318782 20985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.354409 20985 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:40021
I20260810 11:43:51.354460 21124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:40021 every 8 connection(s)
I20260810 11:43:51.355850 20985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:43:51.357331 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 20985
I20260810 11:43:51.357600 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:43:51.358386 20854 external_mini_cluster.cc:1428] Setting key 2544e3b3b60e17d8020e591e30c97be3
Aug 10 11:43:51 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362231, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:43:51.378453 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:51.359023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:40325 (local address 127.20.93.190:36647)
0810 11:43:51.359620 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:51.359626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:51.361658 (+ 2032us) server_negotiation.cc:407] Connection header received
0810 11:43:51.362870 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:51.362878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:51.362978 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:51.363107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:51.364730 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.365906 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.367538 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.367956 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.368113 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:51.372112 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:51.372117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:51.372121 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:51.372158 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:51.375122 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:51.375923 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:51.375929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:51.375935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:51.376077 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:51.376539 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:51.376547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:51.376550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:51.376837 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:51.377002 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:51.377818 (+ 816us) server_negotiation.cc:299] Negotiation successful
0810 11:43:51.378074 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":369,"threads_started":1}
I20260810 11:43:51.379556 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:51.359706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36647 (local address 127.20.93.129:40325)
0810 11:43:51.360596 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:51.360650 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:51.362581 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:51.363346 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:51.363355 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:51.363942 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:51.364544 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:51.364561 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.366100 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.366104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:51.367380 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:51.367392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.368216 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.368224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:51.368476 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:51.369255 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:51.369277 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:51.371850 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:51.375290 (+ 3440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:51.375301 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:51.375319 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:51.375789 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:51.376198 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:51.376203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:51.376207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:51.376419 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:51.376976 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:51.376984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:51.377371 (+ 387us) client_negotiation.cc:769] Sending connection context
0810 11:43:51.377708 (+ 337us) client_negotiation.cc:240] Negotiation successful
0810 11:43:51.378296 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":385,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:43:51.381251 21125 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36647
I20260810 11:43:51.381637 21125 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:51.382490 21125 heartbeater.cc:511] Master 127.20.93.190:36647 requested a full tablet report, sending...
I20260810 11:43:51.384872 20914 ts_manager.cc:194] Registered new tserver with Master: 5056e332357645d79cea21a07ef1194e (127.20.93.129:40021)
I20260810 11:43:51.386451 20914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:40325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:43:51.405049 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:51.752085 21132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:51.752406 21132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:51.752511 21132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:51.760511 21132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:51.760655 21132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:51.760788 21132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:43:51.771498 21132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:51.772914 21132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:51.774334 21132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:51.788908 21141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:51.788909 21138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:51.789047 21139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:51.790840 21132 server_base.cc:1034] running on GCE node
I20260810 11:43:51.791806 21132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:51.792882 21132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:51.794134 21132 hybrid_clock.cc:648] HybridClock initialized: now 1786362231794064 us; error 63 us; skew 500 ppm
I20260810 11:43:51.794502 21132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:51 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362231, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:51.797905 21132 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:43:51.799574 21132 webserver.cc:460] Webserver started at http://127.20.93.130:45749/ using document root <none> and password file <none>
I20260810 11:43:51.800349 21132 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:51.800510 21132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:51.800853 21132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:51.804064 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "11df5b947ed848e5ab426361a62a2d3c"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "3f87874ad8588e7cd2154856ec7f07ea"
server_key_iv: "e2f1ad0dc8e413a746ec9e5a29baa13b"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.804891 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "11df5b947ed848e5ab426361a62a2d3c"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "3f87874ad8588e7cd2154856ec7f07ea"
server_key_iv: "e2f1ad0dc8e413a746ec9e5a29baa13b"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.810794 21132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 11:43:51.814770 21149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.816227 21132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:43:51.816449 21132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "11df5b947ed848e5ab426361a62a2d3c"
format_stamp: "Formatted at 2026-08-10 11:43:51 on dist-test-slave-njxd"
server_key: "3f87874ad8588e7cd2154856ec7f07ea"
server_key_iv: "e2f1ad0dc8e413a746ec9e5a29baa13b"
server_key_version: "encryptionkey@0"
I20260810 11:43:51.816680 21132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:51.836112 21132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:51.840672 21132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:51.841043 21132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:51.842144 21132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:51.843593 21132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:51.843708 21132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.843863 21132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:51.843940 21132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:51.881464 21132 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:36401
I20260810 11:43:51.881556 21276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:36401 every 8 connection(s)
I20260810 11:43:51.882864 21132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:43:51.886148 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21132
I20260810 11:43:51.886371 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:43:51.886962 20854 external_mini_cluster.cc:1428] Setting key 15adad60f272a456f83f627cc6552dc0
Aug 10 11:43:51 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362231, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:51.907869 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:51.885711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:59651 (local address 127.20.93.190:36647)
0810 11:43:51.886259 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:51.886269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:51.887502 (+ 1233us) server_negotiation.cc:407] Connection header received
0810 11:43:51.889910 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:51.889915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:51.889981 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:51.890088 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:51.892432 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.893202 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.895790 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.896284 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.896465 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:51.900290 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:51.900300 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:51.900308 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:51.900366 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:51.904015 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:51.904886 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:51.904893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:51.904896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:51.905076 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:51.905760 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:51.905769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:51.905792 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:51.906229 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:51.906412 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:51.907138 (+ 726us) server_negotiation.cc:299] Negotiation successful
0810 11:43:51.907422 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260810 11:43:51.908600 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:51.886314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36647 (local address 127.20.93.130:59651)
0810 11:43:51.887302 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:51.887357 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:51.889555 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:51.890383 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:51.890397 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:51.891295 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:51.892231 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:51.892264 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.893374 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.893382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:51.895593 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:51.895626 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:51.896535 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:51.896540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:51.896738 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:51.897482 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:51.897505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:51.900034 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:51.904170 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:51.904177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:51.904190 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:51.904722 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:51.905243 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:51.905248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:51.905251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:51.905566 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:51.906378 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:51.906384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:51.906738 (+ 354us) client_negotiation.cc:769] Sending connection context
0810 11:43:51.906974 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 11:43:51.907384 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":348,"threads_started":1}
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
I20260810 11:43:51.910254 21277 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36647
I20260810 11:43:51.910598 21277 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:51.911345 21277 heartbeater.cc:511] Master 127.20.93.190:36647 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
I20260810 11:43:51.913144 20914 ts_manager.cc:194] Registered new tserver with Master: 11df5b947ed848e5ab426361a62a2d3c (127.20.93.130:36401)
I20260810 11:43:51.914534 20914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:59651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:43:51.938817 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:52.283085 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:52.283428 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:52.283600 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:52.291416 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:52.291587 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:52.291779 21284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:43:52.301959 21284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:52.303341 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:52.304749 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:52.318830 21289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:52.318883 21290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:52.318913 21292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:52.319679 21284 server_base.cc:1034] running on GCE node
I20260810 11:43:52.320382 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:52.321476 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:52.322686 21284 hybrid_clock.cc:648] HybridClock initialized: now 1786362232322646 us; error 39 us; skew 500 ppm
I20260810 11:43:52.323035 21284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:52.326748 21284 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:43:52.328438 21284 webserver.cc:460] Webserver started at http://127.20.93.131:44313/ using document root <none> and password file <none>
I20260810 11:43:52.329144 21284 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:52.329252 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:52.329531 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:52.332643 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "75aa0f6c228b4056aa76a4694184e5e4"
format_stamp: "Formatted at 2026-08-10 11:43:52 on dist-test-slave-njxd"
server_key: "573863211307d9953a0fbf1e5593e6fb"
server_key_iv: "9e59025a1e060afa489ba1f6bd058d74"
server_key_version: "encryptionkey@0"
I20260810 11:43:52.333395 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "75aa0f6c228b4056aa76a4694184e5e4"
format_stamp: "Formatted at 2026-08-10 11:43:52 on dist-test-slave-njxd"
server_key: "573863211307d9953a0fbf1e5593e6fb"
server_key_iv: "9e59025a1e060afa489ba1f6bd058d74"
server_key_version: "encryptionkey@0"
I20260810 11:43:52.339038 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:43:52.342797 21303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:52.344265 21284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 11:43:52.344462 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "75aa0f6c228b4056aa76a4694184e5e4"
format_stamp: "Formatted at 2026-08-10 11:43:52 on dist-test-slave-njxd"
server_key: "573863211307d9953a0fbf1e5593e6fb"
server_key_iv: "9e59025a1e060afa489ba1f6bd058d74"
server_key_version: "encryptionkey@0"
I20260810 11:43:52.344683 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:52.377879 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:52.382504 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:52.382839 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:52.384025 21284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:52.385637 21284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:52.385738 21284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:52.385917 21284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:52.385978 21284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:52.389743 21125 heartbeater.cc:503] Master 127.20.93.190:36647 was elected leader, sending a full tablet report...
I20260810 11:43:52.422060 21284 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:32809
I20260810 11:43:52.422104 21423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:32809 every 8 connection(s)
I20260810 11:43:52.423452 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:43:52.430019 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21284
I20260810 11:43:52.430227 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:43:52.430681 20854 external_mini_cluster.cc:1428] Setting key 7d12490b392df3bf102595347fb9ccd1
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:52.444854 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.426125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:41389 (local address 127.20.93.190:36647)
0810 11:43:52.426595 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.426600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.427502 (+ 902us) server_negotiation.cc:407] Connection header received
0810 11:43:52.429383 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.429389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.429523 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.429680 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.431534 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.432797 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.435157 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.435538 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.435637 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.439330 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.439335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.439339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.439371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.442056 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.442799 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.442802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.442804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.442921 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.443351 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.443354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.443356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.443625 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.443748 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.444398 (+ 650us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.444563 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":253,"threads_started":1}
I20260810 11:43:52.446175 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.426615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36647 (local address 127.20.93.131:41389)
0810 11:43:52.427337 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:52.427375 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:52.429098 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:52.429951 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:52.429960 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:52.430651 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:52.431322 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:52.431342 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.432957 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.432961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:52.435014 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:52.435028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.435768 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.435773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:52.435987 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.436648 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:52.436670 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:52.439113 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:52.442210 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:52.442218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:52.442232 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:52.442693 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:52.443041 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:52.443045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:52.443047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:52.443249 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:52.443745 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:52.443751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:52.444076 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 11:43:52.444313 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 11:43:52.444717 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":293,"threads_started":1}
I20260810 11:43:52.447723 21424 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36647
I20260810 11:43:52.448081 21424 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:52.448856 21424 heartbeater.cc:511] Master 127.20.93.190:36647 requested a full tablet report, sending...
I20260810 11:43:52.450464 20913 ts_manager.cc:194] Registered new tserver with Master: 75aa0f6c228b4056aa76a4694184e5e4 (127.20.93.131:32809)
I20260810 11:43:52.451366 20913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:41389
I20260810 11:43:52.460033 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:43:52.510699 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.495683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.20.93.129:40021)
0810 11:43:52.496366 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.496374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.496403 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:43:52.496509 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.496517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.496754 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.496889 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.498350 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.499344 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.500648 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.501263 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.501362 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.505235 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.505248 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.505263 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.505304 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.508108 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.508766 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.508773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.508775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.508896 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.509211 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.509214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.509216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.509566 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.509877 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.510070 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.510278 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":343,"threads_started":1}
W20260810 11:43:52.512086 21031 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50564
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:43:52.562798 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.548215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.20.93.129:40021)
0810 11:43:52.548472 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.548479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.548744 (+ 265us) server_negotiation.cc:407] Connection header received
0810 11:43:52.548882 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.548886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.548985 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.549073 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.550538 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.551213 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.552396 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.552831 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.552924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.556853 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.556858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.556862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.556901 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.559768 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.560607 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.560612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.560614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.560775 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.561212 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.561215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.561217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.561506 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.561793 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.562216 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.562415 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:43:52.564671 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 20985
I20260810 11:43:52.578382 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.565827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.20.93.129:40021)
0810 11:43:52.566036 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.566044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.566355 (+ 311us) server_negotiation.cc:407] Connection header received
0810 11:43:52.566538 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.566543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.566617 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.566719 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.567953 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.568687 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.570062 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.570481 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.570625 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.572943 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.572956 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.572962 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.572993 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.575644 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.576334 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.576341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.576344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.576489 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.576877 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.576886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.576889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.577168 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.577428 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.577785 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.578014 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 11:43:52.579214 21120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:52.663913 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 20985
I20260810 11:43:52.682683 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21132
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:43:52.699616 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.683572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.20.93.130:36401)
0810 11:43:52.684119 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.684126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.684153 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:43:52.684223 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.684230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.684432 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.684569 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.685985 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.687011 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.688368 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.688952 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.689047 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.693758 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.693770 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.693785 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.693823 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.696635 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.697325 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.697333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.697335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.697461 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.697978 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.697985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.697988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.698347 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.698688 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.698975 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.699253 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":296,"threads_started":1}
I20260810 11:43:52.700958 21272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:52.786757 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21132
I20260810 11:43:52.805909 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21284
Aug 10 11:43:52 dist-test-slave-njxd krb5kdc[20862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:43:52.822741 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:52.806957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.20.93.131:32809)
0810 11:43:52.807576 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:52.807583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:52.807631 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:43:52.807690 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:52.807697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:52.807904 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:52.808114 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:52.809448 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.810549 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.811872 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:52.812457 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:52.812607 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:52.817017 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:52.817027 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:52.817041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:52.817075 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:52.819835 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.820523 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.820530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.820532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.820674 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:52.821046 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:52.821051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:52.821054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:52.821418 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:52.821787 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:52.822089 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 11:43:52.822374 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":310,"threads_started":1}
I20260810 11:43:52.824100 21418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:52.908484 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21284
I20260810 11:43:52.927688 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 20877
I20260810 11:43:52.928966 20948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:53.034632 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 20877
2026-08-10T11:43:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4218 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:43:53 dist-test-slave-njxd krb5kdc[21446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:43:53 dist-test-slave-njxd krb5kdc[21446](info): set up 2 sockets
Aug 10 11:43:53 dist-test-slave-njxd krb5kdc[21446](info): commencing operation
krb5kdc: starting...
W20260810 11:43:54.176628 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:43:54 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:43:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:43:54.381511 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35497
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:54.713469 21460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:54.713864 21460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:54.713968 21460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:54.721624 21460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:43:54.721756 21460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:54.721804 21460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:54.721861 21460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:43:54.721905 21460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:43:54.732419 21460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:54.733852 21460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:54.735446 21460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:54.748320 21466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:54.748293 21465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:54.748497 21468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:54.749199 21460 server_base.cc:1034] running on GCE node
I20260810 11:43:54.750061 21460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:54.751449 21460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:54.752667 21460 hybrid_clock.cc:648] HybridClock initialized: now 1786362234752586 us; error 59 us; skew 500 ppm
I20260810 11:43:54.753027 21460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:54 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362234, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:54.760214 21460 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:43:54.761840 21460 webserver.cc:460] Webserver started at http://127.20.93.190:37209/ using document root <none> and password file <none>
I20260810 11:43:54.762564 21460 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:54.762701 21460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:54.763046 21460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:54.766179 21460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "076b3d56bee14ee8934eaab4339b3800"
format_stamp: "Formatted at 2026-08-10 11:43:54 on dist-test-slave-njxd"
server_key: "234c8275150474c88655484666561725"
server_key_iv: "aa8e601c22a58357f5b5bea1b1191da2"
server_key_version: "encryptionkey@0"
I20260810 11:43:54.766966 21460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "076b3d56bee14ee8934eaab4339b3800"
format_stamp: "Formatted at 2026-08-10 11:43:54 on dist-test-slave-njxd"
server_key: "234c8275150474c88655484666561725"
server_key_iv: "aa8e601c22a58357f5b5bea1b1191da2"
server_key_version: "encryptionkey@0"
I20260810 11:43:54.772509 21460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:43:54.775998 21477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:54.777362 21460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:43:54.777565 21460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "076b3d56bee14ee8934eaab4339b3800"
format_stamp: "Formatted at 2026-08-10 11:43:54 on dist-test-slave-njxd"
server_key: "234c8275150474c88655484666561725"
server_key_iv: "aa8e601c22a58357f5b5bea1b1191da2"
server_key_version: "encryptionkey@0"
I20260810 11:43:54.777854 21460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:54.796754 21460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:54.801323 21460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:54.801710 21460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:54.821432 21460 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35497
I20260810 11:43:54.821465 21532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35497 every 8 connection(s)
I20260810 11:43:54.822868 21460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:43:54.826711 21533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:43:54.831434 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21460
I20260810 11:43:54.831701 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:43:54.832556 20854 external_mini_cluster.cc:1428] Setting key 0966a85f3f2e5ee2ac7f626c4c7c3d0f
I20260810 11:43:54.838721 21533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800: Bootstrap starting.
I20260810 11:43:54.841989 21533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:54.843047 21533 log.cc:824] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:54.846226 21533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800: No bootstrap required, opened a new log
Aug 10 11:43:54 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:54.855597 21533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } }
I20260810 11:43:54.856005 21533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:54.856145 21533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076b3d56bee14ee8934eaab4339b3800, State: Initialized, Role: FOLLOWER
I20260810 11:43:54.856554 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:54.834843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55142 (local address 127.20.93.190:35497)
0810 11:43:54.835665 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:54.835677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:54.835722 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:43:54.837492 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:54.837521 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:54.837976 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:54.838366 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:54.839885 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:54.841201 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:54.842607 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:54.843341 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:54.843527 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:54.848553 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:54.848561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:54.848579 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:54.848627 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:54.851931 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:54.852814 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:54.852820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:54.852827 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:54.852991 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:54.853421 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:54.853424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:54.853426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:54.854037 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:54.854278 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:54.854880 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 11:43:54.855286 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":378,"threads_started":1}
I20260810 11:43:54.856789 21533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } }
I20260810 11:43:54.857086 21533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:43:54.857247 21533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:43:54.857450 21533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:54.860527 21533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } }
I20260810 11:43:54.861037 21533 leader_election.cc:302] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 076b3d56bee14ee8934eaab4339b3800; no voters:
I20260810 11:43:54.861678 21533 leader_election.cc:288] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:43:54.861784 21538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:43:54.863025 21538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [term 1 LEADER]: Becoming Leader. State: Replica: 076b3d56bee14ee8934eaab4339b3800, State: Running, Role: LEADER
I20260810 11:43:54.863623 21538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } }
I20260810 11:43:54.863931 21533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:43:54.866585 21539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "076b3d56bee14ee8934eaab4339b3800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } } }
I20260810 11:43:54.866668 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 076b3d56bee14ee8934eaab4339b3800. Latest consensus state: current_term: 1 leader_uuid: "076b3d56bee14ee8934eaab4339b3800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076b3d56bee14ee8934eaab4339b3800" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35497 } } }
I20260810 11:43:54.867017 21539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:54.867067 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800 [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:54.867975 21546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:43:54.874728 21546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:43:54.884243 21546 catalog_manager.cc:1347] Generated new cluster ID: f20df485e1784e6c98c8123ae4e0f361
I20260810 11:43:54.884368 21546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:43:54.911157 21546 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:43:54.913167 21546 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:43:54.923698 21546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 076b3d56bee14ee8934eaab4339b3800: Generated new TSK 0
I20260810 11:43:54.924624 21546 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:43:54.983750 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:55.326996 21559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:55.327276 21559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:55.327371 21559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:55.334491 21559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:55.334615 21559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:55.334740 21559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:43:55.344857 21559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:55.346208 21559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:55.347504 21559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:55.360803 21568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:55.360832 21566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:55.360726 21565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:55.361923 21559 server_base.cc:1034] running on GCE node
I20260810 11:43:55.362541 21559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:55.363479 21559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:55.364737 21559 hybrid_clock.cc:648] HybridClock initialized: now 1786362235364688 us; error 43 us; skew 500 ppm
I20260810 11:43:55.365108 21559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:55 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:55.368534 21559 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:43:55.370113 21559 webserver.cc:460] Webserver started at http://127.20.93.129:34817/ using document root <none> and password file <none>
I20260810 11:43:55.370885 21559 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:55.371019 21559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:55.371351 21559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:55.374444 21559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "eda81c4f87b24866abe3f99ba73b5aaa"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "93b4d8d8989d3079738df0754cd7ed47"
server_key_iv: "ae717454a28a9165a1c9222fc466aad4"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.375286 21559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "eda81c4f87b24866abe3f99ba73b5aaa"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "93b4d8d8989d3079738df0754cd7ed47"
server_key_iv: "ae717454a28a9165a1c9222fc466aad4"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.380832 21559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:43:55.384936 21576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.386401 21559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:43:55.386579 21559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "eda81c4f87b24866abe3f99ba73b5aaa"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "93b4d8d8989d3079738df0754cd7ed47"
server_key_iv: "ae717454a28a9165a1c9222fc466aad4"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.386801 21559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:55.406566 21559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:55.411168 21559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:55.411487 21559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:55.412519 21559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:55.413977 21559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:55.414055 21559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.414222 21559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:55.414283 21559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.448599 21559 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:43339
I20260810 11:43:55.448642 21699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:43339 every 8 connection(s)
I20260810 11:43:55.450048 21559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:43:55.454521 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21559
I20260810 11:43:55.454766 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:43:55.455592 20854 external_mini_cluster.cc:1428] Setting key b99ef2f2b2b71a5359a7da5f66fdc76d
Aug 10 11:43:55 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:55.474426 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:55.452759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:34721 (local address 127.20.93.190:35497)
0810 11:43:55.453246 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:55.453251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:55.454346 (+ 1095us) server_negotiation.cc:407] Connection header received
0810 11:43:55.456759 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:55.456765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:55.456862 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:55.456996 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:55.459155 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:55.460136 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:55.462811 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:55.463221 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:55.463326 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:55.467778 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:55.467784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:55.467787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:55.467823 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:55.470538 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:55.471641 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:55.471646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:55.471648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:55.471769 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:55.472303 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:55.472311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:55.472314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:55.472626 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:55.472741 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:55.473842 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0810 11:43:55.474104 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260810 11:43:55.476111 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:55.453200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35497 (local address 127.20.93.129:34721)
0810 11:43:55.454002 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:55.454058 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:55.456461 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:55.457266 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:55.457275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:55.458078 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:55.458952 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:55.458982 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:55.460346 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:55.460380 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:55.462648 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:55.462674 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:55.463466 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:55.463475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:55.463830 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:55.464652 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:55.464696 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:55.467511 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:55.470729 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:55.470747 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:55.470769 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:55.471509 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:55.471879 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:55.471886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:55.471893 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:55.472152 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:55.472767 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:55.472778 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:55.473294 (+ 516us) client_negotiation.cc:769] Sending connection context
0810 11:43:55.473745 (+ 451us) client_negotiation.cc:240] Negotiation successful
0810 11:43:55.474400 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":334,"threads_started":1}
I20260810 11:43:55.477967 21700 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35497
I20260810 11:43:55.478349 21700 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:43:55.479208 21700 heartbeater.cc:511] Master 127.20.93.190:35497 requested a full tablet report, sending...
I20260810 11:43:55.481675 21495 ts_manager.cc:194] Registered new tserver with Master: eda81c4f87b24866abe3f99ba73b5aaa (127.20.93.129:43339)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:43:55.483982 21495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:34721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:43:55.509367 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:55.846416 21706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:55.846748 21706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:55.846920 21706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:55.854455 21706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:55.854624 21706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:55.854810 21706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:43:55.864542 21706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:55.865931 21706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:55.867283 21706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:55.880909 21711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:55.880873 21712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:55.880950 21716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:55.881944 21706 server_base.cc:1034] running on GCE node
I20260810 11:43:55.882656 21706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:55.883811 21706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:55.885035 21706 hybrid_clock.cc:648] HybridClock initialized: now 1786362235884933 us; error 92 us; skew 500 ppm
I20260810 11:43:55.885389 21706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:55 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:55.888921 21706 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:43:55.890591 21706 webserver.cc:460] Webserver started at http://127.20.93.130:40391/ using document root <none> and password file <none>
I20260810 11:43:55.891319 21706 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:55.891486 21706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:55.891820 21706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:55.894853 21706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "2391fd461a8044578472e3fef01e6324"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "0808197b0d2a57e86c64073b6075d531"
server_key_iv: "adea8e481652ab9d282747e48a6df047"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.895607 21706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "2391fd461a8044578472e3fef01e6324"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "0808197b0d2a57e86c64073b6075d531"
server_key_iv: "adea8e481652ab9d282747e48a6df047"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.900754 21706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:43:55.904219 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.905555 21706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:43:55.905814 21706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "2391fd461a8044578472e3fef01e6324"
format_stamp: "Formatted at 2026-08-10 11:43:55 on dist-test-slave-njxd"
server_key: "0808197b0d2a57e86c64073b6075d531"
server_key_iv: "adea8e481652ab9d282747e48a6df047"
server_key_version: "encryptionkey@0"
I20260810 11:43:55.906024 21706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:55.945389 21706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:55.949693 21706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:55.949995 21706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:55.951011 21706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:55.952827 21706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:55.952921 21706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.953092 21706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:55.953150 21706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:55.987986 21706 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42687
I20260810 11:43:55.988077 21839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42687 every 8 connection(s)
I20260810 11:43:55.989383 21706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:43:55.991220 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21706
I20260810 11:43:55.991447 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:43:55.992120 20854 external_mini_cluster.cc:1428] Setting key 2222335127007dc2464e2d114a5fff1b
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
I20260810 11:43:56.014272 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:55.992869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:47919 (local address 127.20.93.190:35497)
0810 11:43:55.993408 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:55.993415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:55.994773 (+ 1358us) server_negotiation.cc:407] Connection header received
0810 11:43:55.996765 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:55.996773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:55.996882 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:55.997010 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:55.998710 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:55.999860 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.002166 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.002627 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.002748 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.007265 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.007270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.007273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.007312 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.010426 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.011556 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.011560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.011563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.011704 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.012277 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.012280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.012282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.012572 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.012685 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.013793 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.013981 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":285,"threads_started":1}
I20260810 11:43:56.015394 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:55.993692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35497 (local address 127.20.93.130:47919)
0810 11:43:55.994586 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:55.994646 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:55.996461 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:55.997237 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:55.997246 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:55.997910 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:55.998527 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:55.998545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.000027 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.000031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.001998 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.002016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.002885 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.002895 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.003256 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.004032 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:56.004067 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:56.006984 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:56.010648 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:56.010662 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:56.010685 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:56.011385 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:56.011844 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:56.011851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:56.011857 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:56.012159 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:56.012733 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:56.012746 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:56.013272 (+ 526us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.013673 (+ 401us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.014171 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":381,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
I20260810 11:43:56.017170 21840 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35497
I20260810 11:43:56.017556 21840 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:56.018580 21840 heartbeater.cc:511] Master 127.20.93.190:35497 requested a full tablet report, sending...
I20260810 11:43:56.020442 21495 ts_manager.cc:194] Registered new tserver with Master: 2391fd461a8044578472e3fef01e6324 (127.20.93.130:42687)
I20260810 11:43:56.021320 21495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:47919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:43:56.042898 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:56.384706 21847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:56.385003 21847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:56.385185 21847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:56.392715 21847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:56.392838 21847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:56.393054 21847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:43:56.403476 21847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:56.404907 21847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:56.406342 21847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:56.419811 21853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:56.419759 21856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:56.419768 21854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:56.421034 21847 server_base.cc:1034] running on GCE node
I20260810 11:43:56.421772 21847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:56.422735 21847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:56.423920 21847 hybrid_clock.cc:648] HybridClock initialized: now 1786362236423858 us; error 58 us; skew 500 ppm
I20260810 11:43:56.424253 21847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:56.427664 21847 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:43:56.429251 21847 webserver.cc:460] Webserver started at http://127.20.93.131:34359/ using document root <none> and password file <none>
I20260810 11:43:56.430045 21847 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:56.430192 21847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:56.430524 21847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:56.433473 21847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "da980d28062a4221870532b60ce5dea1"
format_stamp: "Formatted at 2026-08-10 11:43:56 on dist-test-slave-njxd"
server_key: "7d01eb83059c7711f1480a00bec9cb9d"
server_key_iv: "c6db746fd0b7ed7198c4dc94d535dcd2"
server_key_version: "encryptionkey@0"
I20260810 11:43:56.434365 21847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "da980d28062a4221870532b60ce5dea1"
format_stamp: "Formatted at 2026-08-10 11:43:56 on dist-test-slave-njxd"
server_key: "7d01eb83059c7711f1480a00bec9cb9d"
server_key_iv: "c6db746fd0b7ed7198c4dc94d535dcd2"
server_key_version: "encryptionkey@0"
I20260810 11:43:56.439575 21847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:43:56.443039 21864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:56.444418 21847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:43:56.444609 21847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "da980d28062a4221870532b60ce5dea1"
format_stamp: "Formatted at 2026-08-10 11:43:56 on dist-test-slave-njxd"
server_key: "7d01eb83059c7711f1480a00bec9cb9d"
server_key_iv: "c6db746fd0b7ed7198c4dc94d535dcd2"
server_key_version: "encryptionkey@0"
I20260810 11:43:56.444835 21847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:56.464205 21847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:56.468830 21847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:56.469143 21847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:56.470207 21847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:43:56.471598 21847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:56.471752 21847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:43:56.471900 21847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:56.471951 21847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:56.487792 21700 heartbeater.cc:503] Master 127.20.93.190:35497 was elected leader, sending a full tablet report...
I20260810 11:43:56.507757 21847 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:45925
I20260810 11:43:56.507802 21989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:45925 every 8 connection(s)
I20260810 11:43:56.509126 21847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:43:56.513415 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 21847
I20260810 11:43:56.513670 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:43:56.514245 20854 external_mini_cluster.cc:1428] Setting key 572bc1a92fb65d3bdb62202a94e3e1b7
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:56.531152 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.511931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:59709 (local address 127.20.93.190:35497)
0810 11:43:56.512110 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.512115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.513551 (+ 1436us) server_negotiation.cc:407] Connection header received
0810 11:43:56.516195 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.516199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.516269 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.516402 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.518289 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.519052 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.520603 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.520995 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.521093 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.524662 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.524667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.524671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.524707 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.527520 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.528420 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.528425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.528428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.528605 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.529165 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.529170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.529173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.529518 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.529770 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.530705 (+ 935us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.530896 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:43:56.532205 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.512423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35497 (local address 127.20.93.131:59709)
0810 11:43:56.513315 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.513370 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.515862 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.516705 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.516717 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.517347 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:56.518131 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.518150 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.519189 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.519192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.520453 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.520466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.521195 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.521199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.521399 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.522110 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:56.522139 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:56.524416 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:56.527653 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:56.527660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:56.527674 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:56.528271 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:56.528760 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:56.528764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:56.528767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:56.529026 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:56.529756 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:56.529764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:56.530214 (+ 450us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.530597 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.531051 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":305,"threads_started":1}
I20260810 11:43:56.533623 21990 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35497
I20260810 11:43:56.533926 21990 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:56.534652 21990 heartbeater.cc:511] Master 127.20.93.190:35497 requested a full tablet report, sending...
I20260810 11:43:56.536043 21495 ts_manager.cc:194] Registered new tserver with Master: da980d28062a4221870532b60ce5dea1 (127.20.93.131:45925)
I20260810 11:43:56.536928 21495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:59709
I20260810 11:43:56.542667 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:43:56.562095 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.549219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.20.93.190:35497)
0810 11:43:56.549376 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.549381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.550003 (+ 622us) server_negotiation.cc:407] Connection header received
0810 11:43:56.550140 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.550144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.550215 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.550302 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.551788 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.552505 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.553788 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.554195 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.554291 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.556502 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.556507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.556510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.556544 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.559444 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.560138 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.560141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.560144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.560281 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.560685 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.560688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.560690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.561016 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.561206 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.561655 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.561840 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 11:43:56.568858 21495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:43:56.576193 21495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:43:56.601910 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.590588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.20.93.131:45925)
0810 11:43:56.592356 (+ 1768us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.592367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.592407 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:43:56.592522 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.592532 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.592856 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.593093 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:43:56.593983 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.596243 (+ 2260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.599630 (+ 3387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.600681 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.600835 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.601103 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.601359 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.601525 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1552,"thread_start_us":1364,"threads_started":1}
I20260810 11:43:56.602289 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.590165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:45925 (local address 127.0.0.1:60452)
0810 11:43:56.590699 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.590719 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.590834 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.593113 (+ 2279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.593118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.593156 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:43:56.593721 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.593737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.596452 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.596460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.599420 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.599434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.600898 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.600904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.601105 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.601171 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.601311 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.601511 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260810 11:43:56.603004 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.588558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42687 (local address 127.0.0.1:37728)
0810 11:43:56.589671 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.589707 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.589902 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.591009 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.591016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.591057 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:43:56.591560 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.591581 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.596206 (+ 4625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.596214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.599728 (+ 3514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.599742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.602053 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.602059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.602250 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.602283 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.602361 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.602496 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":4,"thread_start_us":726,"threads_started":1}
I20260810 11:43:56.603830 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.588805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.20.93.130:42687)
0810 11:43:56.589470 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.589477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.589816 (+ 339us) server_negotiation.cc:407] Connection header received
0810 11:43:56.590200 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.590213 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.590569 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.590792 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:43:56.593711 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.595991 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.599929 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.601541 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.601681 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.601873 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.603227 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.603420 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":371,"threads_started":1}
I20260810 11:43:56.606554 21914 tablet_service.cc:1467] Processing CreateTablet for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2 (DEFAULT_TABLE table=test-table [id=4f0c7068c30d4d0ca0b054d9effe9cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:43:56.607649 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.591035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.20.93.129:43339)
0810 11:43:56.598523 (+ 7488us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.598532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.599186 (+ 654us) server_negotiation.cc:407] Connection header received
0810 11:43:56.599496 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.599504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.599773 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.599953 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:43:56.600871 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.602459 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.604816 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.606305 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.606448 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.606671 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.607148 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.607316 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7301,"thread_start_us":7151,"threads_started":1}
I20260810 11:43:56.608175 21914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:43:56.607774 21772 tablet_service.cc:1467] Processing CreateTablet for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2 (DEFAULT_TABLE table=test-table [id=4f0c7068c30d4d0ca0b054d9effe9cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:43:56.608367 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.591002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:43339 (local address 127.0.0.1:51554)
0810 11:43:56.599067 (+ 8065us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.599085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.599299 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.600250 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.600255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.600287 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:43:56.600655 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.600666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.602645 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.602650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.604651 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.604664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.606533 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.606541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.606930 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.606962 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.607908 (+ 946us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.608041 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7931,"thread_start_us":7759,"threads_started":1}
I20260810 11:43:56.609200 21772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:43:56.612098 21629 tablet_service.cc:1467] Processing CreateTablet for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2 (DEFAULT_TABLE table=test-table [id=4f0c7068c30d4d0ca0b054d9effe9cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:43:56.613469 21629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:43:56.621713 22011 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Bootstrap starting.
I20260810 11:43:56.623296 22012 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Bootstrap starting.
I20260810 11:43:56.626960 22011 tablet_bootstrap.cc:654] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:56.626960 22012 tablet_bootstrap.cc:654] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:56.627938 22013 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Bootstrap starting.
I20260810 11:43:56.628418 22012 log.cc:824] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:56.628418 22011 log.cc:824] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:56.631958 22013 tablet_bootstrap.cc:654] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:56.632253 22011 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: No bootstrap required, opened a new log
I20260810 11:43:56.632660 22012 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: No bootstrap required, opened a new log
I20260810 11:43:56.632669 22011 ts_tablet_manager.cc:1397] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 11:43:56.633003 22012 ts_tablet_manager.cc:1397] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 11:43:56.633522 22013 log.cc:824] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:56.637248 22013 tablet_bootstrap.cc:492] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: No bootstrap required, opened a new log
I20260810 11:43:56.637624 22013 ts_tablet_manager.cc:1397] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 11:43:56.641120 22011 raft_consensus.cc:348] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.641745 22011 raft_consensus.cc:374] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:56.641884 22011 raft_consensus.cc:729] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2391fd461a8044578472e3fef01e6324, State: Initialized, Role: FOLLOWER
I20260810 11:43:56.642711 22011 consensus_queue.cc:260] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.643931 21840 heartbeater.cc:503] Master 127.20.93.190:35497 was elected leader, sending a full tablet report...
I20260810 11:43:56.644018 22012 raft_consensus.cc:348] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.644481 22012 raft_consensus.cc:374] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:56.644575 22012 raft_consensus.cc:729] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da980d28062a4221870532b60ce5dea1, State: Initialized, Role: FOLLOWER
I20260810 11:43:56.644713 22011 ts_tablet_manager.cc:1428] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260810 11:43:56.645243 22012 consensus_queue.cc:260] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.646546 21990 heartbeater.cc:503] Master 127.20.93.190:35497 was elected leader, sending a full tablet report...
I20260810 11:43:56.647198 22012 ts_tablet_manager.cc:1428] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260810 11:43:56.648880 22013 raft_consensus.cc:348] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.649626 22013 raft_consensus.cc:374] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:56.649747 22013 raft_consensus.cc:729] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda81c4f87b24866abe3f99ba73b5aaa, State: Initialized, Role: FOLLOWER
I20260810 11:43:56.650504 22013 consensus_queue.cc:260] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.652503 22013 ts_tablet_manager.cc:1428] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260810 11:43:56.656699 22019 raft_consensus.cc:482] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:43:56.656952 22019 raft_consensus.cc:504] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.658633 22019 leader_election.cc:288] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2391fd461a8044578472e3fef01e6324 (127.20.93.130:42687), da980d28062a4221870532b60ce5dea1 (127.20.93.131:45925)
I20260810 11:43:56.666205 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.658897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42687 (local address 127.20.93.129:50143)
0810 11:43:56.659454 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.659471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.659701 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.660181 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.660184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.660206 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:43:56.660598 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.660613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.661944 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.661950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.663767 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.663781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.665188 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.665194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.665434 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.665459 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.665567 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.665762 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":56,"thread_start_us":292,"threads_started":1}
I20260810 11:43:56.666510 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.659054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:50143 (local address 127.20.93.130:42687)
0810 11:43:56.659320 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.659327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.659624 (+ 297us) server_negotiation.cc:407] Connection header received
0810 11:43:56.659865 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.659873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.659965 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.660066 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:43:56.660767 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.661818 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.663937 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.664976 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.665160 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.665249 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.665909 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.666050 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:43:56.666581 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.659757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:33471 (local address 127.20.93.131:45925)
0810 11:43:56.659987 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.659995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.660411 (+ 416us) server_negotiation.cc:407] Connection header received
0810 11:43:56.660547 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.660554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.660632 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.660727 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:43:56.661329 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.662234 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.664180 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.665083 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.665264 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.665351 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.666158 (+ 807us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.666313 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:43:56.667289 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.659649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:45925 (local address 127.20.93.129:33471)
0810 11:43:56.660299 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:56.660313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:56.660421 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:56.660767 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:56.660772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:56.660797 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:43:56.661180 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.661191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.662381 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.662384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.663984 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:56.664000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.665750 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.665754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:56.665941 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.665971 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:43:56.666763 (+ 792us) client_negotiation.cc:240] Negotiation successful
0810 11:43:56.666890 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":2,"thread_start_us":326,"threads_started":1}
I20260810 11:43:56.667799 21792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94f2eaa6f1f4b3d84c11f0b199c54f2" candidate_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2391fd461a8044578472e3fef01e6324" is_pre_election: true
I20260810 11:43:56.668470 21792 raft_consensus.cc:2454] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda81c4f87b24866abe3f99ba73b5aaa in term 0.
I20260810 11:43:56.668344 21939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94f2eaa6f1f4b3d84c11f0b199c54f2" candidate_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da980d28062a4221870532b60ce5dea1" is_pre_election: true
I20260810 11:43:56.668912 21939 raft_consensus.cc:2454] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda81c4f87b24866abe3f99ba73b5aaa in term 0.
I20260810 11:43:56.669292 21579 leader_election.cc:302] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2391fd461a8044578472e3fef01e6324, eda81c4f87b24866abe3f99ba73b5aaa; no voters:
I20260810 11:43:56.669818 22019 raft_consensus.cc:2781] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:43:56.669971 22019 raft_consensus.cc:482] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:43:56.670105 22019 raft_consensus.cc:3037] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:56.672545 22019 raft_consensus.cc:504] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.673156 22019 leader_election.cc:288] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [CANDIDATE]: Term 1 election: Requested vote from peers 2391fd461a8044578472e3fef01e6324 (127.20.93.130:42687), da980d28062a4221870532b60ce5dea1 (127.20.93.131:45925)
I20260810 11:43:56.673746 21792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94f2eaa6f1f4b3d84c11f0b199c54f2" candidate_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2391fd461a8044578472e3fef01e6324"
I20260810 11:43:56.673894 21939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94f2eaa6f1f4b3d84c11f0b199c54f2" candidate_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da980d28062a4221870532b60ce5dea1"
I20260810 11:43:56.673982 21792 raft_consensus.cc:3037] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:56.674117 21939 raft_consensus.cc:3037] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:56.677487 21792 raft_consensus.cc:2454] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda81c4f87b24866abe3f99ba73b5aaa in term 1.
I20260810 11:43:56.677487 21939 raft_consensus.cc:2454] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda81c4f87b24866abe3f99ba73b5aaa in term 1.
I20260810 11:43:56.677937 21579 leader_election.cc:302] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2391fd461a8044578472e3fef01e6324, eda81c4f87b24866abe3f99ba73b5aaa; no voters:
I20260810 11:43:56.678212 22019 raft_consensus.cc:2781] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:43:56.678681 22019 raft_consensus.cc:686] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [term 1 LEADER]: Becoming Leader. State: Replica: eda81c4f87b24866abe3f99ba73b5aaa, State: Running, Role: LEADER
I20260810 11:43:56.679185 22019 consensus_queue.cc:237] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } }
I20260810 11:43:56.684180 21494 catalog_manager.cc:5495] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa reported cstate change: term changed from 0 to 1, leader changed from <none> to eda81c4f87b24866abe3f99ba73b5aaa (127.20.93.129). New cstate: current_term: 1 leader_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2391fd461a8044578472e3fef01e6324" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da980d28062a4221870532b60ce5dea1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 45925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda81c4f87b24866abe3f99ba73b5aaa" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 43339 } health_report { overall_health: HEALTHY } } }
W20260810 11:43:56.702750 21701 tablet.cc:2369] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P eda81c4f87b24866abe3f99ba73b5aaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:43:56.720666 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.705105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.20.93.129:43339)
0810 11:43:56.705356 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.705387 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.706140 (+ 753us) server_negotiation.cc:407] Connection header received
0810 11:43:56.706367 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.706377 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.706464 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.706578 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.708150 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.708901 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.710347 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.710781 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.710932 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.715162 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.715177 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.715194 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.715233 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.718106 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.718837 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.718845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.718847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.719014 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.719353 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.719356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.719358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.719705 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.719947 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.720220 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.720411 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 11:43:56.741816 21842 tablet.cc:2369] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P 2391fd461a8044578472e3fef01e6324: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260810 11:43:56.761029 21991 tablet.cc:2369] T f94f2eaa6f1f4b3d84c11f0b199c54f2 P da980d28062a4221870532b60ce5dea1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:43:56.774624 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.760151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.20.93.129:43339)
0810 11:43:56.760394 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.760403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.760540 (+ 137us) server_negotiation.cc:407] Connection header received
0810 11:43:56.760739 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.760745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.760836 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.760957 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.762456 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.763192 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.764533 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.764946 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.765076 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.768915 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.768920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.768924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.768958 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.771721 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.772470 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.772474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.772476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.772615 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.773045 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.773050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.773054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.773359 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.773723 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.773997 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.774250 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:43:56.828303 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.813821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.20.93.129:43339)
0810 11:43:56.814056 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.814064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.814308 (+ 244us) server_negotiation.cc:407] Connection header received
0810 11:43:56.814532 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.814536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.814623 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.814715 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.816063 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.816809 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.818063 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.818505 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.818600 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.822953 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.822958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.822961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.822996 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.825796 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.826550 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.826555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.826557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.826690 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.827049 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.827052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.827054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.827318 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.827519 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.827852 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.828015 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:43:56.830225 21608 tablet_service.cc:2908] Rejecting scan request for tablet f94f2eaa6f1f4b3d84c11f0b199c54f2: Uninitialized: clean time has not yet been initialized
I20260810 11:43:56.833503 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21559
I20260810 11:43:56.846208 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.834540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.20.93.129:43339)
0810 11:43:56.834731 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.834737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.835103 (+ 366us) server_negotiation.cc:407] Connection header received
0810 11:43:56.835227 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.835232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.835303 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.835391 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.836518 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.837645 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.838777 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.839185 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.839277 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.841407 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.841412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.841415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.841445 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.844128 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.844685 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.844689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.844691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.844804 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.845099 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.845102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.845104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.845319 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.845487 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.845777 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.845956 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:43:56.846861 21693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:56.942237 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21559
I20260810 11:43:56.962332 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21706
Aug 10 11:43:56 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:43:56.979147 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:56.963376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.20.93.130:42687)
0810 11:43:56.963566 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:56.963571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:56.963610 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:43:56.963669 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:56.963674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:56.963764 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:56.963868 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:56.965247 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.966159 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.967534 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:56.967932 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:56.968029 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:56.972513 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:56.972529 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:56.972547 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:56.972589 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:56.975930 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.976708 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.976717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.976719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.976863 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:56.977273 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:56.977277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:56.977279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:56.977938 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:56.978237 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:56.978518 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 11:43:56.978705 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:43:56.979909 21835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:57.071620 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21706
I20260810 11:43:57.093148 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21847
Aug 10 11:43:57 dist-test-slave-njxd krb5kdc[21446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:43:57.109040 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:57.093977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.20.93.131:45925)
0810 11:43:57.094184 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:57.094189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:57.094209 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:43:57.094325 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:57.094330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:57.094410 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:57.094496 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:57.095806 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:57.096561 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:57.098046 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:57.098445 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:57.098543 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:57.103007 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:57.103023 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:57.103040 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:57.103080 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:57.106123 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:57.106853 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:57.106861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:57.106863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:57.107006 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:57.107410 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:57.107417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:57.107420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:57.107824 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:57.108126 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:57.108448 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 11:43:57.108713 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:43:57.109746 21985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:57.201053 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21847
I20260810 11:43:57.219529 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 21460
I20260810 11:43:57.220765 21527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:43:57.332075 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 21460
2026-08-10T11:43:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4294 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:43:57 dist-test-slave-njxd krb5kdc[22042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:43:57 dist-test-slave-njxd krb5kdc[22042](info): set up 2 sockets
Aug 10 11:43:57 dist-test-slave-njxd krb5kdc[22042](info): commencing operation
krb5kdc: starting...
W20260810 11:43:58.480507 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:43:58 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:43:58.701292 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34877
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:34877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:59.056005 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:59.056334 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:59.056514 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:59.064886 22058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:43:59.065028 22058 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:43:59.065140 22058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:59.065263 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:59.065358 22058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:43:59.065454 22058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:43:59.078661 22058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:34877
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:59.080058 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:59.081475 22058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:59.094151 22063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:59.094218 22064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:59.094206 22066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:59.094945 22058 server_base.cc:1034] running on GCE node
I20260810 11:43:59.095810 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:59.097281 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:59.098575 22058 hybrid_clock.cc:648] HybridClock initialized: now 1786362239098485 us; error 62 us; skew 500 ppm
I20260810 11:43:59.098968 22058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:59 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:59.102324 22058 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:43:59.103863 22058 webserver.cc:460] Webserver started at http://127.20.93.190:41677/ using document root <none> and password file <none>
I20260810 11:43:59.104547 22058 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:59.104638 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:59.104935 22058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:59.108100 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "980fc72e79144242af99c207e2e844bd"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "547d18cc4914085d3fd3e2959346bf9e"
server_key_iv: "63bf4e57fe3bfac43417f69867fa09b3"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.108984 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "980fc72e79144242af99c207e2e844bd"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "547d18cc4914085d3fd3e2959346bf9e"
server_key_iv: "63bf4e57fe3bfac43417f69867fa09b3"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.114822 22058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 11:43:59.118820 22073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:59.120352 22058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:43:59.120554 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "980fc72e79144242af99c207e2e844bd"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "547d18cc4914085d3fd3e2959346bf9e"
server_key_iv: "63bf4e57fe3bfac43417f69867fa09b3"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.120810 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:59.151856 22058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:59.156522 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:59.156831 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:59.179663 22058 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:34877
I20260810 11:43:59.179682 22137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:34877 every 8 connection(s)
I20260810 11:43:59.181129 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:43:59.183414 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22058
I20260810 11:43:59.183735 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:43:59.184638 20854 external_mini_cluster.cc:1428] Setting key 7e5732e6633e227715f9c8bfb96c95b4
I20260810 11:43:59.186257 22138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:43:59 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:59.201279 22138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Bootstrap starting.
I20260810 11:43:59.204700 22138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Neither blocks nor log segments found. Creating new log.
I20260810 11:43:59.206096 22138 log.cc:824] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Log is configured to *not* fsync() on all Append() calls
I20260810 11:43:59.208837 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:59.186896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.20.93.190:34877)
0810 11:43:59.187719 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:59.187731 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:59.187774 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:43:59.189395 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:59.189417 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:59.189902 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:59.190255 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:59.192041 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.193416 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.195209 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.195888 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.196023 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:59.201109 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:59.201116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:59.201133 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:59.201171 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:59.204388 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.205290 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.205297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.205303 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.205469 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.205956 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.205959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.205961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.206573 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:59.206950 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:59.207416 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 11:43:59.207822 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":413,"threads_started":1}
I20260810 11:43:59.209393 22138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: No bootstrap required, opened a new log
I20260810 11:43:59.217871 22138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } }
I20260810 11:43:59.218394 22138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:43:59.218535 22138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980fc72e79144242af99c207e2e844bd, State: Initialized, Role: FOLLOWER
I20260810 11:43:59.219206 22138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } }
I20260810 11:43:59.219461 22138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:43:59.219609 22138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:43:59.219810 22138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:43:59.222126 22138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } }
I20260810 11:43:59.222648 22138 leader_election.cc:302] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 980fc72e79144242af99c207e2e844bd; no voters:
I20260810 11:43:59.223335 22138 leader_election.cc:288] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:43:59.223516 22145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:43:59.224440 22145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [term 1 LEADER]: Becoming Leader. State: Replica: 980fc72e79144242af99c207e2e844bd, State: Running, Role: LEADER
I20260810 11:43:59.224995 22145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } }
I20260810 11:43:59.225802 22138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:43:59.228220 22146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "980fc72e79144242af99c207e2e844bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } } }
I20260810 11:43:59.228561 22148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980fc72e79144242af99c207e2e844bd. Latest consensus state: current_term: 1 leader_uuid: "980fc72e79144242af99c207e2e844bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980fc72e79144242af99c207e2e844bd" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34877 } } }
I20260810 11:43:59.228725 22146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:59.228830 22148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd [sys.catalog]: This master's current role is: LEADER
I20260810 11:43:59.230842 22155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:43:59.236276 22155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:43:59.244963 22155 catalog_manager.cc:1347] Generated new cluster ID: 84628a69f801428da48b5d19ab73561e
I20260810 11:43:59.245093 22155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:43:59.262607 22155 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:43:59.264657 22155 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:43:59.272433 22155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Generated new TSK 0
I20260810 11:43:59.273362 22155 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:43:59.336910 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34877
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:43:59.709057 22168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:43:59.709435 22168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:43:59.709681 22168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:43:59.717233 22168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:43:59.717418 22168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:43:59.717644 22168 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:43:59.717787 22168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:43:59.728576 22168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:34877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:43:59.730082 22168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:43:59.731698 22168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:43:59.745251 22177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:59.745450 22175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:43:59.746189 22174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:43:59.747839 22168 server_base.cc:1034] running on GCE node
I20260810 11:43:59.748741 22168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:43:59.750030 22168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:43:59.751835 22168 hybrid_clock.cc:648] HybridClock initialized: now 1786362239751762 us; error 35 us; skew 500 ppm
I20260810 11:43:59.752161 22168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:43:59 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:43:59.756265 22168 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:43:59.758498 22168 webserver.cc:460] Webserver started at http://127.20.93.129:45035/ using document root <none> and password file <none>
I20260810 11:43:59.759509 22168 fs_manager.cc:359] Metadata directory not provided
I20260810 11:43:59.759647 22168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:43:59.760035 22168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:43:59.764571 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "520103e2be1e460a8ac3fa51aecf1cf1"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "d52481cecb10df4597a00d184430fb89"
server_key_iv: "c2b41a5ba3bbf5866b190f1db0a43b88"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.765856 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "520103e2be1e460a8ac3fa51aecf1cf1"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "d52481cecb10df4597a00d184430fb89"
server_key_iv: "c2b41a5ba3bbf5866b190f1db0a43b88"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.773922 22168 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260810 11:43:59.779268 22184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:59.780857 22168 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:43:59.781033 22168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "520103e2be1e460a8ac3fa51aecf1cf1"
format_stamp: "Formatted at 2026-08-10 11:43:59 on dist-test-slave-njxd"
server_key: "d52481cecb10df4597a00d184430fb89"
server_key_iv: "c2b41a5ba3bbf5866b190f1db0a43b88"
server_key_version: "encryptionkey@0"
I20260810 11:43:59.781271 22168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:43:59.802929 22168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:43:59.806955 22168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:43:59.807189 22168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:43:59.809980 22168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:43:59.810097 22168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:43:59.810231 22168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:43:59.810308 22168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:43:59 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:43:59.838424 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:59.810896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:39251 (local address 127.20.93.190:34877)
0810 11:43:59.811417 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:59.811435 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:59.812461 (+ 1026us) server_negotiation.cc:407] Connection header received
0810 11:43:59.814930 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:59.814936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:59.815045 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:59.815194 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:59.817917 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.819206 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.822080 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.822716 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.822893 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:59.828793 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:59.828798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:59.828801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:59.828837 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:59.833164 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.834225 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.834231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.834235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.834399 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.835115 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.835123 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.835129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.835571 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:59.835754 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:59.837838 (+ 2084us) server_negotiation.cc:299] Negotiation successful
0810 11:43:59.838093 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":301,"threads_started":1}
I20260810 11:43:59.839035 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:59.811453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.129:39251)
0810 11:43:59.812296 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:59.812335 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:59.814544 (+ 2209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:59.815594 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:59.815602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:59.816465 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:59.817483 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:59.817506 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.819436 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.819445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:59.821858 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:59.821876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.823276 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.823287 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:59.823685 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:59.824786 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:59.824827 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:59.828506 (+ 3679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:59.833338 (+ 4832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:59.833346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:59.833359 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:59.834084 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:59.834555 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:59.834561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:59.834565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:59.834929 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:59.835747 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:59.835758 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:59.836204 (+ 446us) client_negotiation.cc:769] Sending connection context
0810 11:43:59.836635 (+ 431us) client_negotiation.cc:240] Negotiation successful
0810 11:43:59.837326 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":360,"threads_started":1}
I20260810 11:43:59.858157 22168 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:35823
I20260810 11:43:59.858189 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:35823 every 8 connection(s)
I20260810 11:43:59.859511 22168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:43:59.862478 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:59.847090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:39851 (local address 127.20.93.190:34877)
0810 11:43:59.847294 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 11:43:59.847302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:43:59.848105 (+ 803us) server_negotiation.cc:407] Connection header received
0810 11:43:59.848452 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:43:59.848461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:43:59.848563 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:43:59.848694 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:43:59.850613 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.851619 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.852980 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.853610 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.853763 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:59.856403 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:43:59.856408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:43:59.856411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:43:59.856446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:43:59.859430 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.860190 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.860197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.860201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.860351 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:43:59.860876 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:43:59.860882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:43:59.860885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:43:59.861240 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:43:59.861428 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:43:59.861981 (+ 553us) server_negotiation.cc:299] Negotiation successful
0810 11:43:59.862252 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:43:59.862071 22320 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34877
I20260810 11:43:59.862507 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:43:59.847116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.129:39851)
0810 11:43:59.847990 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 11:43:59.848006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:43:59.848193 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:43:59.848898 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:43:59.848908 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:43:59.849773 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:43:59.850338 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:59.850363 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.851774 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.851787 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:59.852834 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:43:59.852846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:43:59.853849 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:43:59.853859 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:43:59.854135 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:43:59.855129 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:43:59.855162 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:43:59.856165 (+ 1003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:43:59.859590 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:59.859596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:59.859600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:59.860057 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:59.860536 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:43:59.860543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:43:59.860550 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:43:59.860728 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:43:59.861395 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:43:59.861400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:43:59.861682 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 11:43:59.861861 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 11:43:59.862191 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":520,"threads_started":1}
I20260810 11:43:59.863175 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22168
I20260810 11:43:59.863394 22320 heartbeater.cc:464] Registering TS with master...
I20260810 11:43:59.863430 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:43:59.864110 20854 external_mini_cluster.cc:1428] Setting key ff0eabe4e13af56fbd8a27326e1ad1a3
I20260810 11:43:59.864287 22320 heartbeater.cc:511] Master 127.20.93.190:34877 requested a full tablet report, sending...
I20260810 11:43:59.867684 22091 ts_manager.cc:194] Registered new tserver with Master: 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129:35823)
I20260810 11:43:59.870019 22091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:39251
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:43:59.913673 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34877
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:00.263403 22324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:00.263722 22324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:00.263825 22324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:00.271245 22324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:00.271379 22324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:00.271466 22324 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:00.271549 22324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:00.282271 22324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:34877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:00.283603 22324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:00.284957 22324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:00.299240 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:00.299341 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:00.299270 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:00.303032 22324 server_base.cc:1034] running on GCE node
I20260810 11:44:00.303735 22324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:00.304754 22324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:00.306017 22324 hybrid_clock.cc:648] HybridClock initialized: now 1786362240305944 us; error 35 us; skew 500 ppm
I20260810 11:44:00.306515 22324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:00 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:00.309935 22324 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:00.311439 22324 webserver.cc:460] Webserver started at http://127.20.93.130:35677/ using document root <none> and password file <none>
I20260810 11:44:00.312216 22324 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:00.312335 22324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:00.312677 22324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:00.315766 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "b00f3ab4a3e8b7b47a8653d565f64bc6"
server_key_iv: "20b7dd5298e09817e9e5a586e508f3b3"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.316592 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "b00f3ab4a3e8b7b47a8653d565f64bc6"
server_key_iv: "20b7dd5298e09817e9e5a586e508f3b3"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.322125 22324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:44:00.325774 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:00.327246 22324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:00.327437 22324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "b00f3ab4a3e8b7b47a8653d565f64bc6"
server_key_iv: "20b7dd5298e09817e9e5a586e508f3b3"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.327675 22324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:00.347635 22324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:00.352455 22324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:00.352773 22324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:00.355680 22324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:00.355818 22324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:00.355962 22324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:00.356050 22324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:00 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:00.382227 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.356588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:55973 (local address 127.20.93.190:34877)
0810 11:44:00.356778 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:00.356785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:00.358441 (+ 1656us) server_negotiation.cc:407] Connection header received
0810 11:44:00.361016 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:00.361023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:00.361129 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:00.361266 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:00.364504 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.365508 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.368020 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.368528 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.368680 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.373762 (+ 5082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:00.373773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:00.373777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:00.373813 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:00.377424 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.378639 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.378646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.378652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.378828 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.379586 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.379590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.379592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.380023 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:00.380187 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:00.381339 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0810 11:44:00.381700 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:44:00.383029 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.357236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.130:55973)
0810 11:44:00.358235 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:00.358288 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:00.360558 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:00.361656 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:00.361678 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:00.362769 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:00.364246 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.364289 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.365737 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.365751 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.367867 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.367880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.368855 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.368867 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.369289 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.370342 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:00.370385 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:00.373472 (+ 3087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:00.377659 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.377668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.377692 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.378416 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.378955 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.378960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.378969 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.379378 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.380163 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:00.380176 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:00.380756 (+ 580us) client_negotiation.cc:769] Sending connection context
0810 11:44:00.381162 (+ 406us) client_negotiation.cc:240] Negotiation successful
0810 11:44:00.381749 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":434,"threads_started":1}
I20260810 11:44:00.409258 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.392776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:36833 (local address 127.20.93.190:34877)
0810 11:44:00.392964 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:00.392969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:00.394297 (+ 1328us) server_negotiation.cc:407] Connection header received
0810 11:44:00.394524 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:00.394533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:00.394633 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:00.394748 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:00.396443 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.397236 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.399218 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.399725 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.399872 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.402512 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:00.402519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:00.402524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:00.402569 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:00.406177 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.407178 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.407182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.407184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.407319 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.407722 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.407726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.407728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.408062 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:00.408228 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:00.408657 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 11:44:00.408896 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:44:00.409397 22432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.393217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.130:36833)
0810 11:44:00.394177 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:00.394194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:00.394372 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:00.394932 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:00.394935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:00.395734 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:00.396224 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.396245 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.397376 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.397382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.399036 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.399059 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.400102 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.400107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.400226 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.401000 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:00.401030 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:00.402238 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:00.406375 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.406380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.406385 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.406992 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.407443 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.407448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.407451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.407588 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.408190 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:00.408195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:00.408398 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 11:44:00.408548 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 11:44:00.408905 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":487,"threads_started":1}
I20260810 11:44:00.412784 22324 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44625
I20260810 11:44:00.413113 22472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44625 every 8 connection(s)
I20260810 11:44:00.414311 22324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:00.416431 22473 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34877
I20260810 11:44:00.416632 22473 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:00.417227 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22324
I20260810 11:44:00.417227 22473 heartbeater.cc:511] Master 127.20.93.190:34877 requested a full tablet report, sending...
I20260810 11:44:00.417443 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:00.418098 20854 external_mini_cluster.cc:1428] Setting key 9a25109e89c29d9e50ac79ff4fdc61ec
I20260810 11:44:00.418826 22091 ts_manager.cc:194] Registered new tserver with Master: cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625)
I20260810 11:44:00.419673 22091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:55973
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:00.465976 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34877
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:00.810525 22479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:00.810873 22479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:00.811045 22479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:00.818521 22479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:00.818693 22479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:00.818840 22479 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:00.818979 22479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:00.830940 22479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:34877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:00.832408 22479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:00.833891 22479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:00.847898 22485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:00.847940 22487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:00.848176 22484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:00.849442 22479 server_base.cc:1034] running on GCE node
I20260810 11:44:00.850128 22479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:00.851024 22479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:00.852283 22479 hybrid_clock.cc:648] HybridClock initialized: now 1786362240852214 us; error 45 us; skew 500 ppm
I20260810 11:44:00.852598 22479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:00 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:00.855834 22479 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:00.857448 22479 webserver.cc:460] Webserver started at http://127.20.93.131:36385/ using document root <none> and password file <none>
I20260810 11:44:00.858208 22479 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:00.858340 22479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:00.858681 22479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:00.861681 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "7ad9d0d365b9a5a540c01f67b56cd3b5"
server_key_iv: "568e862961ebab7761f87c65540e4a0b"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.862534 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "7ad9d0d365b9a5a540c01f67b56cd3b5"
server_key_iv: "568e862961ebab7761f87c65540e4a0b"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.867952 22479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:00.871645 22495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:00.872977 22479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:00.873080 22320 heartbeater.cc:503] Master 127.20.93.190:34877 was elected leader, sending a full tablet report...
I20260810 11:44:00.873172 22479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
format_stamp: "Formatted at 2026-08-10 11:44:00 on dist-test-slave-njxd"
server_key: "7ad9d0d365b9a5a540c01f67b56cd3b5"
server_key_iv: "568e862961ebab7761f87c65540e4a0b"
server_key_version: "encryptionkey@0"
I20260810 11:44:00.873452 22479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:00.890938 22479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:00.895236 22479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:00.895525 22479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:00.898430 22479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:00.898551 22479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:00.898690 22479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:00.898763 22479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:00 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:00.923822 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.898802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:42967 (local address 127.20.93.190:34877)
0810 11:44:00.899018 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:00.899024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:00.900504 (+ 1480us) server_negotiation.cc:407] Connection header received
0810 11:44:00.902704 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:00.902710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:00.902813 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:00.902927 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:00.905290 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.906735 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.909462 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.910120 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.910291 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.915544 (+ 5253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:00.915552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:00.915560 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:00.915612 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:00.919351 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.920602 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.920608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.920612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.920770 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.921257 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.921262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.921266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.921554 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:00.921770 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:00.922913 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0810 11:44:00.923257 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:44:00.924538 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.899546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.131:42967)
0810 11:44:00.900339 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:00.900381 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:00.902395 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:00.903245 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:00.903259 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:00.904151 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:00.905030 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.905079 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.906932 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.906939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.909283 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.909297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.910399 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.910407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.910651 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.911598 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:00.911621 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:00.915243 (+ 3622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:00.919586 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.919598 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.919624 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.920445 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.920900 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.920905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.920908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.921125 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.921755 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:00.921764 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:00.922356 (+ 592us) client_negotiation.cc:769] Sending connection context
0810 11:44:00.922720 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 11:44:00.923267 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":349,"threads_started":1}
I20260810 11:44:00.949810 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.933030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:60961 (local address 127.20.93.190:34877)
0810 11:44:00.933218 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:00.933223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:00.933946 (+ 723us) server_negotiation.cc:407] Connection header received
0810 11:44:00.934164 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:00.934169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:00.934260 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:00.934368 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:00.935376 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.936099 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.937961 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.938606 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.938776 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.941861 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:00.941871 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:00.941878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:00.941926 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:00.946039 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.947150 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.947155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.947157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.947351 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:00.947916 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:00.947925 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:00.947929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:00.948263 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:00.948455 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:00.949163 (+ 708us) server_negotiation.cc:299] Negotiation successful
0810 11:44:00.949414 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:44:00.950196 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:00.933127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.20.93.131:60961)
0810 11:44:00.933837 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:00.933856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:00.934022 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:00.934397 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:00.934401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:00.934925 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:00.935217 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.935226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.936273 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.936277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.937802 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:00.937814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:00.938881 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:00.938894 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:00.939192 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:00.939990 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:00.940023 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:00.941488 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:00.946227 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.946236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.946245 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.946978 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.947511 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:00.947516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:00.947520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:00.947741 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:00.948409 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:00.948421 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:00.948776 (+ 355us) client_negotiation.cc:769] Sending connection context
0810 11:44:00.948983 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 11:44:00.949462 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":425,"threads_started":1}
I20260810 11:44:00.956991 22479 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:33205
I20260810 11:44:00.957031 22621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:33205 every 8 connection(s)
I20260810 11:44:00.958336 22479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:00.960533 22622 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34877
I20260810 11:44:00.960793 22622 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:00.961468 22622 heartbeater.cc:511] Master 127.20.93.190:34877 requested a full tablet report, sending...
I20260810 11:44:00.962886 22091 ts_manager.cc:194] Registered new tserver with Master: eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:00.963704 22091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:42967
I20260810 11:44:00.967995 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22479
I20260810 11:44:00.968163 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:00.968555 20854 external_mini_cluster.cc:1428] Setting key 50f3faf94f938f8f6aea354d9f46f99f
I20260810 11:44:00.971707 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:44:00 dist-test-slave-njxd krb5kdc[22042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:44:01 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:01.025282 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.010418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.20.93.190:34877)
0810 11:44:01.010684 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.010693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.010968 (+ 275us) server_negotiation.cc:407] Connection header received
0810 11:44:01.011196 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.011201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.011279 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.011399 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:01.012877 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.013744 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.015104 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.015556 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.015702 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.019756 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:01.019761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:01.019764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:01.019804 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:01.022534 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:01.023215 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:01.023221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:01.023225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:01.023369 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:01.023768 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:01.023774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:01.023777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:01.024068 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:01.024347 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.024774 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.024966 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:44:01.030155 22091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:01.037524 22091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:01.076421 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.052786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35823 (local address 127.0.0.1:41754)
0810 11:44:01.066192 (+ 13406us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.066228 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.066382 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.067055 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.067059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.067082 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:01.067846 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.067873 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.069771 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.069776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.072851 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.072870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.075266 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.075273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.075523 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.075589 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.075734 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.075956 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13254,"mutex_wait_us":170,"thread_start_us":12674,"threads_started":1}
I20260810 11:44:01.076895 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.050439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.20.93.130:44625)
0810 11:44:01.066643 (+ 16204us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.066652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.066680 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:44:01.066786 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.066793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.067005 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.067219 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:01.067989 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.069555 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.073522 (+ 3967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.075261 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.075955 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.076159 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.076246 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.076432 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16050,"thread_start_us":15929,"threads_started":1}
I20260810 11:44:01.077450 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.051748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.20.93.131:33205)
0810 11:44:01.066225 (+ 14477us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.066235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.066274 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:44:01.066402 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.066413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.066702 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.066954 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:01.068514 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.069925 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.073320 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.074865 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.075009 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.075249 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.076823 (+ 1574us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.077113 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14308,"thread_start_us":14244,"threads_started":1}
I20260810 11:44:01.077870 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.050269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44625 (local address 127.0.0.1:59542)
0810 11:44:01.052396 (+ 2127us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.052461 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.052677 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.067193 (+ 14516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.067197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.067220 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:01.067668 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.067683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.069770 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.069776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.073255 (+ 3479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.073271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.075506 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.075512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.075720 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.075794 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.075854 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.076975 (+ 1121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1757,"mutex_wait_us":1132,"thread_start_us":1590,"threads_started":1}
I20260810 11:44:01.078027 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.053380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.20.93.129:35823)
0810 11:44:01.065750 (+ 12370us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.065763 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.066515 (+ 752us) server_negotiation.cc:407] Connection header received
0810 11:44:01.066626 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.066642 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.066824 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.067645 (+ 821us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:01.068062 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.069637 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.073067 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.074741 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.074882 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.075110 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.077461 (+ 2351us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.077688 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12197,"thread_start_us":12034,"threads_started":1}
I20260810 11:44:01.078235 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.051051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:33205 (local address 127.0.0.1:44046)
0810 11:44:01.065176 (+ 14125us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.065202 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.065359 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.066990 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.067001 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.067037 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:01.068245 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.068259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.070132 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.070140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.073095 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.073115 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.075406 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.075416 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.075899 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.075933 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.076031 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.076168 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13938,"mutex_wait_us":312,"thread_start_us":14809,"threads_started":1}
I20260810 11:44:01.082031 22242 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 (DEFAULT_TABLE table=test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.082518 22548 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 (DEFAULT_TABLE table=test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.083509 22242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7c5836e2214bc5a1806260fc13d5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.083092 22396 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 (DEFAULT_TABLE table=test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.083979 22548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7c5836e2214bc5a1806260fc13d5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.084448 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7c5836e2214bc5a1806260fc13d5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.097694 22643 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Bootstrap starting.
I20260810 11:44:01.099135 22644 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Bootstrap starting.
I20260810 11:44:01.100242 22645 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Bootstrap starting.
I20260810 11:44:01.101595 22643 tablet_bootstrap.cc:654] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.102756 22644 tablet_bootstrap.cc:654] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.103045 22643 log.cc:824] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:01.103935 22645 tablet_bootstrap.cc:654] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.103964 22644 log.cc:824] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:01.105113 22645 log.cc:824] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:01.106521 22643 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: No bootstrap required, opened a new log
I20260810 11:44:01.106860 22643 ts_tablet_manager.cc:1397] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 11:44:01.107062 22644 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: No bootstrap required, opened a new log
I20260810 11:44:01.107327 22644 ts_tablet_manager.cc:1397] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 11:44:01.108152 22645 tablet_bootstrap.cc:492] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: No bootstrap required, opened a new log
I20260810 11:44:01.108403 22645 ts_tablet_manager.cc:1397] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 11:44:01.115183 22644 raft_consensus.cc:348] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.115593 22644 raft_consensus.cc:374] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.115691 22644 raft_consensus.cc:729] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc25e9faeeb2479bb81eb2d76552fcb9, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.116354 22644 consensus_queue.cc:260] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.117497 22473 heartbeater.cc:503] Master 127.20.93.190:34877 was elected leader, sending a full tablet report...
I20260810 11:44:01.118242 22644 ts_tablet_manager.cc:1428] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260810 11:44:01.118770 22643 raft_consensus.cc:348] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.119264 22643 raft_consensus.cc:374] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.119385 22643 raft_consensus.cc:729] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.119338 22645 raft_consensus.cc:348] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.119771 22645 raft_consensus.cc:374] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.119877 22645 raft_consensus.cc:729] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb67d2f817c34bbf90ecc0ce9b1f5529, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.120040 22643 consensus_queue.cc:260] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.120432 22645 consensus_queue.cc:260] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.124797 22622 heartbeater.cc:503] Master 127.20.93.190:34877 was elected leader, sending a full tablet report...
I20260810 11:44:01.125842 22645 ts_tablet_manager.cc:1428] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
I20260810 11:44:01.127489 22643 ts_tablet_manager.cc:1428] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent starting tablet: real 0.020s user 0.015s sys 0.003s
W20260810 11:44:01.166383 22476 tablet.cc:2369] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:44:01.210074 22624 tablet.cc:2369] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:44:01.362709 22321 tablet.cc:2369] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:01.421142 22651 raft_consensus.cc:482] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:01.421519 22651 raft_consensus.cc:504] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.423696 22651 leader_election.cc:288] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:01.431408 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.425130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41619 (local address 127.20.93.131:33205)
0810 11:44:01.425424 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.425430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.425452 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:44:01.425530 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.425538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.425681 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.425759 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:01.426304 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.427012 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.429297 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.430223 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.430422 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.430503 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.430989 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.431123 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260810 11:44:01.431524 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.423983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:33205 (local address 127.20.93.129:41619)
0810 11:44:01.425112 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.425144 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.425310 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.425848 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.425851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.425873 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:01.426174 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.426182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.427121 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.427125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.429124 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.429135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.430456 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.430462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.430821 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.430848 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.430957 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.431107 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":90,"thread_start_us":708,"threads_started":1}
I20260810 11:44:01.432844 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7c5836e2214bc5a1806260fc13d5e3" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" is_pre_election: true
I20260810 11:44:01.433337 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.424480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:52085 (local address 127.20.93.130:44625)
0810 11:44:01.424784 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.424794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.425399 (+ 605us) server_negotiation.cc:407] Connection header received
0810 11:44:01.427731 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.427737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.427863 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.427959 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:01.428611 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.429316 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.431162 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.432394 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.432587 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.432664 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.432946 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.433076 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 11:44:01.433360 22569 raft_consensus.cc:2454] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:01.433440 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.424062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44625 (local address 127.20.93.129:52085)
0810 11:44:01.425306 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.425322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.425500 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.427985 (+ 2485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.427990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.428017 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:01.428449 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.428462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.429467 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.429474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.431006 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.431018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.432629 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.432636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.432783 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.432807 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.432909 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.433045 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":49,"thread_start_us":1253,"threads_started":1}
I20260810 11:44:01.434123 22190 leader_election.cc:302] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, eb67d2f817c34bbf90ecc0ce9b1f5529; no voters:
I20260810 11:44:01.434603 22651 raft_consensus.cc:2781] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:01.434494 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7c5836e2214bc5a1806260fc13d5e3" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" is_pre_election: true
I20260810 11:44:01.434777 22651 raft_consensus.cc:482] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:01.434926 22651 raft_consensus.cc:3037] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.435109 22424 raft_consensus.cc:2454] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:01.437815 22651 raft_consensus.cc:504] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.438454 22651 leader_election.cc:288] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Requested vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:01.438941 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7c5836e2214bc5a1806260fc13d5e3" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
I20260810 11:44:01.438968 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7c5836e2214bc5a1806260fc13d5e3" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
I20260810 11:44:01.439174 22424 raft_consensus.cc:3037] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.439173 22569 raft_consensus.cc:3037] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.442504 22424 raft_consensus.cc:2454] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:01.442504 22569 raft_consensus.cc:2454] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:01.443020 22190 leader_election.cc:302] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, eb67d2f817c34bbf90ecc0ce9b1f5529; no voters:
I20260810 11:44:01.443348 22651 raft_consensus.cc:2781] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:01.443828 22651 raft_consensus.cc:686] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 LEADER]: Becoming Leader. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Running, Role: LEADER
I20260810 11:44:01.444327 22651 consensus_queue.cc:237] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.449353 22091 catalog_manager.cc:5495] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:01.532742 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.525513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.20.93.129:35823)
0810 11:44:01.525785 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.525793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.526599 (+ 806us) server_negotiation.cc:407] Connection header received
0810 11:44:01.526975 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.526981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.527099 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.527262 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:01.527752 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.528521 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.530096 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.530513 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.530683 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.531409 (+ 726us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:01.532174 (+ 765us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:01.532304 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.532450 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.532523 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260810 11:44:01.541271 22569 raft_consensus.cc:1261] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Refusing update from remote peer 520103e2be1e460a8ac3fa51aecf1cf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:01.542191 22424 raft_consensus.cc:1261] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Refusing update from remote peer 520103e2be1e460a8ac3fa51aecf1cf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:01.542371 22656 consensus_queue.cc:1035] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:01.543120 22651 consensus_queue.cc:1035] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:01.555087 22660 mvcc.cc:205] Tried to move back new op lower bound from 7316939741343006720 to 7316939740959965184. Current Snapshot: MvccSnapshot[applied={T|T < 7316939741343006720}]
I20260810 11:44:01.577148 22090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:01.577553 22090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:01.581305 22090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Sending DeleteTablet for 3 replicas of tablet 6c7c5836e2214bc5a1806260fc13d5e3
I20260810 11:44:01.582574 22396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:01 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59542
I20260810 11:44:01.582691 22242 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:01 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:41754
I20260810 11:44:01.583521 22548 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7c5836e2214bc5a1806260fc13d5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:01 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260810 11:44:01.583843 22671 tablet_replica.cc:330] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: stopping tablet replica
I20260810 11:44:01.584452 22671 raft_consensus.cc:2229] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:01.585142 22671 raft_consensus.cc:2258] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:01.585908 22672 tablet_replica.cc:330] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: stopping tablet replica
I20260810 11:44:01.585840 22090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 11:44:01.586086 22673 tablet_replica.cc:330] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: stopping tablet replica
W20260810 11:44:01.586566 22090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:01.586624 22673 raft_consensus.cc:2229] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:01.587092 22672 raft_consensus.cc:2229] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:01.587162 22673 raft_consensus.cc:2258] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:01.587893 22672 raft_consensus.cc:2258] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:01.590104 22673 ts_tablet_manager.cc:1905] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:01.590106 22672 ts_tablet_manager.cc:1905] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:01.590106 22671 ts_tablet_manager.cc:1905] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:01.595746 22242 tablet_service.cc:1467] Processing CreateTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 (DEFAULT_TABLE table=test-table [id=7ad1a4d2053949959a41f20df5057618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.596283 22242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d8f6392ee2492dbc13b1970f066da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.597419 22673 ts_tablet_manager.cc:1918] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:01.597549 22672 ts_tablet_manager.cc:1918] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:01.597766 22673 log.cc:1196] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/6c7c5836e2214bc5a1806260fc13d5e3
I20260810 11:44:01.597787 22672 log.cc:1196] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/6c7c5836e2214bc5a1806260fc13d5e3
I20260810 11:44:01.598405 22673 ts_tablet_manager.cc:1939] T 6c7c5836e2214bc5a1806260fc13d5e3 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting consensus metadata
I20260810 11:44:01.598418 22672 ts_tablet_manager.cc:1939] T 6c7c5836e2214bc5a1806260fc13d5e3 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting consensus metadata
I20260810 11:44:01.600462 22077 catalog_manager.cc:4841] TS eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205): tablet 6c7c5836e2214bc5a1806260fc13d5e3 (table test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]) successfully deleted
I20260810 11:44:01.601017 22074 catalog_manager.cc:4841] TS 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129:35823): tablet 6c7c5836e2214bc5a1806260fc13d5e3 (table test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]) successfully deleted
I20260810 11:44:01.601683 22548 tablet_service.cc:1467] Processing CreateTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 (DEFAULT_TABLE table=test-table [id=7ad1a4d2053949959a41f20df5057618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.601662 22396 tablet_service.cc:1467] Processing CreateTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 (DEFAULT_TABLE table=test-table [id=7ad1a4d2053949959a41f20df5057618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:01.602136 22548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d8f6392ee2492dbc13b1970f066da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.602170 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d8f6392ee2492dbc13b1970f066da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.603713 22643 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Bootstrap starting.
I20260810 11:44:01.604312 22671 ts_tablet_manager.cc:1918] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:01.604478 22671 log.cc:1196] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/6c7c5836e2214bc5a1806260fc13d5e3
I20260810 11:44:01.605038 22671 ts_tablet_manager.cc:1939] T 6c7c5836e2214bc5a1806260fc13d5e3 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting consensus metadata
I20260810 11:44:01.605345 22643 tablet_bootstrap.cc:654] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.607103 22074 catalog_manager.cc:4841] TS cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625): tablet 6c7c5836e2214bc5a1806260fc13d5e3 (table test-table [id=0a9cb6fb51b541e7a352bdcfcfd3d68d]) successfully deleted
I20260810 11:44:01.608767 22644 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Bootstrap starting.
I20260810 11:44:01.610067 22645 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Bootstrap starting.
I20260810 11:44:01.610459 22644 tablet_bootstrap.cc:654] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.611852 22645 tablet_bootstrap.cc:654] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.623294 22645 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: No bootstrap required, opened a new log
I20260810 11:44:01.623498 22645 ts_tablet_manager.cc:1397] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260810 11:44:01.623800 22644 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: No bootstrap required, opened a new log
I20260810 11:44:01.623963 22644 ts_tablet_manager.cc:1397] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260810 11:44:01.624531 22643 tablet_bootstrap.cc:492] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: No bootstrap required, opened a new log
I20260810 11:44:01.624761 22643 ts_tablet_manager.cc:1397] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.010s
I20260810 11:44:01.624922 22644 raft_consensus.cc:348] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.625178 22644 raft_consensus.cc:374] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.625289 22644 raft_consensus.cc:729] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc25e9faeeb2479bb81eb2d76552fcb9, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.625639 22645 raft_consensus.cc:348] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.625695 22644 consensus_queue.cc:260] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.625891 22645 raft_consensus.cc:374] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.625994 22645 raft_consensus.cc:729] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb67d2f817c34bbf90ecc0ce9b1f5529, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.626116 22643 raft_consensus.cc:348] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.626508 22643 raft_consensus.cc:374] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.626410 22645 consensus_queue.cc:260] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.626713 22643 raft_consensus.cc:729] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.627153 22643 consensus_queue.cc:260] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.627447 22644 ts_tablet_manager.cc:1428] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:01.628093 22643 ts_tablet_manager.cc:1428] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:01.629014 22645 ts_tablet_manager.cc:1428] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 11:44:01.788225 22656 raft_consensus.cc:482] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:01.788511 22656 raft_consensus.cc:504] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.789244 22656 leader_election.cc:288] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:01.789880 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d8f6392ee2492dbc13b1970f066da7" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" is_pre_election: true
I20260810 11:44:01.789918 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d8f6392ee2492dbc13b1970f066da7" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" is_pre_election: true
I20260810 11:44:01.790150 22424 raft_consensus.cc:2454] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:01.790196 22569 raft_consensus.cc:2454] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:01.790648 22190 leader_election.cc:302] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, eb67d2f817c34bbf90ecc0ce9b1f5529; no voters:
I20260810 11:44:01.790988 22656 raft_consensus.cc:2781] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:01.791091 22656 raft_consensus.cc:482] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:01.791190 22656 raft_consensus.cc:3037] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.794302 22656 raft_consensus.cc:504] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.794983 22656 leader_election.cc:288] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Requested vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:01.795564 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d8f6392ee2492dbc13b1970f066da7" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
I20260810 11:44:01.795629 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d8f6392ee2492dbc13b1970f066da7" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
I20260810 11:44:01.795833 22424 raft_consensus.cc:3037] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.795825 22569 raft_consensus.cc:3037] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:01.798952 22424 raft_consensus.cc:2454] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:01.798954 22569 raft_consensus.cc:2454] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:01.799384 22190 leader_election.cc:302] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, eb67d2f817c34bbf90ecc0ce9b1f5529; no voters:
I20260810 11:44:01.799773 22656 raft_consensus.cc:2781] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:01.799978 22656 raft_consensus.cc:686] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 LEADER]: Becoming Leader. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Running, Role: LEADER
I20260810 11:44:01.800256 22656 consensus_queue.cc:237] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.803156 22090 catalog_manager.cc:5495] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } health_report { overall_health: UNKNOWN } } }
Aug 10 11:44:01 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:01.851517 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.832746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34877 (local address 127.0.0.1:43058)
0810 11:44:01.833520 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:01.833538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:01.833731 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:01.838296 (+ 4565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:01.838300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:01.839049 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:01.839277 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.839287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.840421 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.840427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.841501 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:01.841514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.842389 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.842394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:01.842595 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.843311 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:01.843328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:01.845815 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:01.848846 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:01.848860 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:01.848868 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:01.849287 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:01.849685 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:01.849692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:01.849695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:01.849851 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:01.850388 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:01.850397 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:01.850565 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 11:44:01.850724 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 11:44:01.851028 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":449,"threads_started":1}
I20260810 11:44:01.851529 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:01.832653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.20.93.190:34877)
0810 11:44:01.838006 (+ 5353us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:01.838010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:01.838029 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:44:01.838077 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:01.838080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:01.838138 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:01.838267 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:01.839452 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.840276 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.841694 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:01.842202 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:01.842336 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:01.846028 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:01.846033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:01.846036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:01.846065 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:01.848693 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:01.849415 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:01.849425 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:01.849427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:01.849539 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:01.849979 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:01.849984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:01.849987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:01.850256 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:01.850413 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:01.850826 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 11:44:01.851156 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":5237,"thread_start_us":5115,"threads_started":1}
I20260810 11:44:01.857474 22090 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:44:01.858843 22090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:01.866341 22242 tablet_service.cc:1467] Processing CreateTablet for tablet 8807054ef393485e8d0422bee73571fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=480a54266da040f7b043b5aee45e4350]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:01.867002 22242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8807054ef393485e8d0422bee73571fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.872931 22548 tablet_service.cc:1467] Processing CreateTablet for tablet 8807054ef393485e8d0422bee73571fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=480a54266da040f7b043b5aee45e4350]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:01.873730 22548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8807054ef393485e8d0422bee73571fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.874226 22643 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: Bootstrap starting.
I20260810 11:44:01.876399 22643 tablet_bootstrap.cc:654] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.877105 22396 tablet_service.cc:1467] Processing CreateTablet for tablet 8807054ef393485e8d0422bee73571fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=480a54266da040f7b043b5aee45e4350]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:01.877707 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8807054ef393485e8d0422bee73571fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:01.883668 22645 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: Bootstrap starting.
I20260810 11:44:01.885476 22644 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: Bootstrap starting.
I20260810 11:44:01.886178 22645 tablet_bootstrap.cc:654] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.887415 22644 tablet_bootstrap.cc:654] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:01.887693 22643 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: No bootstrap required, opened a new log
I20260810 11:44:01.887962 22643 ts_tablet_manager.cc:1397] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.006s
I20260810 11:44:01.888969 22643 raft_consensus.cc:348] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.889436 22643 raft_consensus.cc:374] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.889622 22643 raft_consensus.cc:729] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.890086 22643 consensus_queue.cc:260] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.890883 22656 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:01.891261 22656 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:01.892448 22643 ts_tablet_manager.cc:1428] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:01.896855 22644 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: No bootstrap required, opened a new log
I20260810 11:44:01.897408 22644 ts_tablet_manager.cc:1397] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260810 11:44:01.898223 22644 raft_consensus.cc:348] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.898535 22644 raft_consensus.cc:374] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.898658 22644 raft_consensus.cc:729] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc25e9faeeb2479bb81eb2d76552fcb9, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.899030 22644 consensus_queue.cc:260] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.899631 22644 ts_tablet_manager.cc:1428] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 11:44:01.899508 22650 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:01.899984 22650 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:01.901352 22645 tablet_bootstrap.cc:492] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: No bootstrap required, opened a new log
I20260810 11:44:01.901552 22645 ts_tablet_manager.cc:1397] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.005s
I20260810 11:44:01.902295 22645 raft_consensus.cc:348] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.902541 22645 raft_consensus.cc:374] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:01.902629 22645 raft_consensus.cc:729] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb67d2f817c34bbf90ecc0ce9b1f5529, State: Initialized, Role: FOLLOWER
I20260810 11:44:01.902884 22645 consensus_queue.cc:260] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:01.903355 22652 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:01.903669 22652 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:01.904450 22645 ts_tablet_manager.cc:1428] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:02.178529 22656 consensus_queue.cc:1035] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:02.184782 22656 consensus_queue.cc:1035] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:02.344380 22656 raft_consensus.cc:482] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:02.344657 22656 raft_consensus.cc:504] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:02.345449 22656 leader_election.cc:288] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:02.346078 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8807054ef393485e8d0422bee73571fc" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" is_pre_election: true
I20260810 11:44:02.346082 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8807054ef393485e8d0422bee73571fc" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" is_pre_election: true
I20260810 11:44:02.346349 22569 raft_consensus.cc:2454] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:02.346376 22424 raft_consensus.cc:2454] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 0.
I20260810 11:44:02.346855 22190 leader_election.cc:302] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, eb67d2f817c34bbf90ecc0ce9b1f5529; no voters:
I20260810 11:44:02.347159 22656 raft_consensus.cc:2781] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:02.347267 22656 raft_consensus.cc:482] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:02.347406 22656 raft_consensus.cc:3037] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:02.349467 22656 raft_consensus.cc:504] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:02.350198 22656 leader_election.cc:288] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Requested vote from peers cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625), eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205)
I20260810 11:44:02.350699 22424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8807054ef393485e8d0422bee73571fc" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9"
I20260810 11:44:02.350793 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8807054ef393485e8d0422bee73571fc" candidate_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529"
I20260810 11:44:02.350970 22424 raft_consensus.cc:3037] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:02.351013 22569 raft_consensus.cc:3037] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:02.353996 22569 raft_consensus.cc:2454] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:02.353996 22424 raft_consensus.cc:2454] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520103e2be1e460a8ac3fa51aecf1cf1 in term 1.
I20260810 11:44:02.354460 22185 leader_election.cc:302] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520103e2be1e460a8ac3fa51aecf1cf1, cc25e9faeeb2479bb81eb2d76552fcb9; no voters:
I20260810 11:44:02.354817 22656 raft_consensus.cc:2781] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:02.354977 22656 raft_consensus.cc:686] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 LEADER]: Becoming Leader. State: Replica: 520103e2be1e460a8ac3fa51aecf1cf1, State: Running, Role: LEADER
I20260810 11:44:02.355268 22656 consensus_queue.cc:237] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } }
I20260810 11:44:02.356737 22651 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: TxnStatusTablet state changed. Reason: New leader 520103e2be1e460a8ac3fa51aecf1cf1. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.356957 22651 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:02.357654 22697 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:44:02.357990 22697 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:44:02.358474 22089 catalog_manager.cc:5495] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:02.460523 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.451547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35823 (local address 127.0.0.1:44254)
0810 11:44:02.454375 (+ 2828us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:02.454392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:02.454542 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:02.454967 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:02.454971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:02.454981 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:02.455318 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:02.455327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.457144 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.457149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:02.458327 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:02.458340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.459317 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.459321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:02.459507 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.459570 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:02.460026 (+ 456us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:02.460043 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 11:44:02.460103 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 11:44:02.460215 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"thread_start_us":2520,"threads_started":1}
I20260810 11:44:02.460690 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.451636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.20.93.129:35823)
0810 11:44:02.454089 (+ 2453us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.454096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.454482 (+ 386us) server_negotiation.cc:407] Connection header received
0810 11:44:02.454718 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.454724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.454820 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.454935 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:02.455722 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.456995 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.458471 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.459161 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.459306 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.459778 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:02.459926 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:02.460020 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.460181 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.460271 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2306,"thread_start_us":2135,"threads_started":1}
I20260810 11:44:02.465020 22424 raft_consensus.cc:1261] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Refusing update from remote peer 520103e2be1e460a8ac3fa51aecf1cf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:02.465078 22569 raft_consensus.cc:1261] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Refusing update from remote peer 520103e2be1e460a8ac3fa51aecf1cf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:02.465714 22651 consensus_queue.cc:1035] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:02.466020 22656 consensus_queue.cc:1035] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:02.469406 22652 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: TxnStatusTablet state changed. Reason: New leader 520103e2be1e460a8ac3fa51aecf1cf1. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.469700 22652 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:02.470396 22650 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: TxnStatusTablet state changed. Reason: New leader 520103e2be1e460a8ac3fa51aecf1cf1. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.470669 22650 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:02.474493 22651 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.474879 22651 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:02.475652 22656 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.475888 22656 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P 520103e2be1e460a8ac3fa51aecf1cf1: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:02.477046 22695 mvcc.cc:205] Tried to move back new op lower bound from 7316939745130676224 to 7316939744689246208. Current Snapshot: MvccSnapshot[applied={T|T < 7316939745130676224}]
I20260810 11:44:02.479324 22652 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.479559 22652 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P eb67d2f817c34bbf90ecc0ce9b1f5529: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:02.490423 22650 tablet_replica.cc:439] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520103e2be1e460a8ac3fa51aecf1cf1" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35823 } } peers { permanent_uuid: "cc25e9faeeb2479bb81eb2d76552fcb9" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44625 } } peers { permanent_uuid: "eb67d2f817c34bbf90ecc0ce9b1f5529" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33205 } } }
I20260810 11:44:02.490720 22650 tablet_replica.cc:442] T 8807054ef393485e8d0422bee73571fc P cc25e9faeeb2479bb81eb2d76552fcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:02.503165 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.496447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:59623 (local address 127.20.93.129:35823)
0810 11:44:02.496641 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.496647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.497354 (+ 707us) server_negotiation.cc:407] Connection header received
0810 11:44:02.497530 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.497535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.497645 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.497754 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:02.498270 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.499461 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.501077 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.501821 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.501953 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.502368 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:02.502500 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:02.502604 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.502744 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.502832 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:44:02.503211 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.496510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35823 (local address 127.20.93.129:59623)
0810 11:44:02.497265 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:02.497285 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:02.497409 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:02.497772 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:02.497775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:02.497785 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:02.498104 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:02.498112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.499599 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.499602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:02.500932 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:02.500945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.501988 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.501993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:02.502113 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.502171 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:02.502616 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:02.502625 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:44:02.502683 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 11:44:02.502802 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":400,"threads_started":1}
I20260810 11:44:02.591369 22090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:02.591634 22090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:02.595045 22090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 980fc72e79144242af99c207e2e844bd: Sending DeleteTablet for 3 replicas of tablet 35d8f6392ee2492dbc13b1970f066da7
I20260810 11:44:02.596022 22396 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:02 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59542
I20260810 11:44:02.596068 22548 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:02 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260810 11:44:02.597016 22242 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d8f6392ee2492dbc13b1970f066da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:02 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:41754
I20260810 11:44:02.602376 22715 tablet_replica.cc:330] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: stopping tablet replica
I20260810 11:44:02.604408 22715 raft_consensus.cc:2229] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:02.605090 22715 raft_consensus.cc:2258] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:02.606776 22715 ts_tablet_manager.cc:1905] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:02.610997 22713 tablet_replica.cc:330] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: stopping tablet replica
I20260810 11:44:02.611263 22713 raft_consensus.cc:2229] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:02.611567 22713 raft_consensus.cc:2258] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:02.613185 22713 ts_tablet_manager.cc:1905] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:02.615765 22714 tablet_replica.cc:330] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: stopping tablet replica
I20260810 11:44:02.615980 22714 raft_consensus.cc:2229] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:02.616250 22714 raft_consensus.cc:2258] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:02.620477 22715 ts_tablet_manager.cc:1918] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:02.620721 22715 log.cc:1196] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/35d8f6392ee2492dbc13b1970f066da7
I20260810 11:44:02.620796 22713 ts_tablet_manager.cc:1918] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:02.620921 22713 log.cc:1196] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/35d8f6392ee2492dbc13b1970f066da7
I20260810 11:44:02.621434 22715 ts_tablet_manager.cc:1939] T 35d8f6392ee2492dbc13b1970f066da7 P 520103e2be1e460a8ac3fa51aecf1cf1: Deleting consensus metadata
I20260810 11:44:02.621428 22713 ts_tablet_manager.cc:1939] T 35d8f6392ee2492dbc13b1970f066da7 P cc25e9faeeb2479bb81eb2d76552fcb9: Deleting consensus metadata
I20260810 11:44:02.622766 22074 catalog_manager.cc:4841] TS 520103e2be1e460a8ac3fa51aecf1cf1 (127.20.93.129:35823): tablet 35d8f6392ee2492dbc13b1970f066da7 (table test-table [id=7ad1a4d2053949959a41f20df5057618]) successfully deleted
I20260810 11:44:02.623076 22074 catalog_manager.cc:4841] TS cc25e9faeeb2479bb81eb2d76552fcb9 (127.20.93.130:44625): tablet 35d8f6392ee2492dbc13b1970f066da7 (table test-table [id=7ad1a4d2053949959a41f20df5057618]) successfully deleted
I20260810 11:44:02.627151 22714 ts_tablet_manager.cc:1905] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:02.634163 22714 ts_tablet_manager.cc:1918] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:02.634342 22714 log.cc:1196] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/35d8f6392ee2492dbc13b1970f066da7
I20260810 11:44:02.634819 22714 ts_tablet_manager.cc:1939] T 35d8f6392ee2492dbc13b1970f066da7 P eb67d2f817c34bbf90ecc0ce9b1f5529: Deleting consensus metadata
I20260810 11:44:02.636050 22077 catalog_manager.cc:4841] TS eb67d2f817c34bbf90ecc0ce9b1f5529 (127.20.93.131:33205): tablet 35d8f6392ee2492dbc13b1970f066da7 (table test-table [id=7ad1a4d2053949959a41f20df5057618]) successfully deleted
Aug 10 11:44:02 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:02.646929 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.603575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.20.93.129:35823)
0810 11:44:02.603777 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.603785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.617017 (+ 13232us) server_negotiation.cc:407] Connection header received
0810 11:44:02.617404 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.617410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.617541 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.617637 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:02.625810 (+ 8173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.626947 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.631793 (+ 4846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.632510 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.632629 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.639451 (+ 6822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:02.639467 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:02.639484 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:02.639523 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:02.643279 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.644094 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.644102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.644105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.644306 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.644768 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.644772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.644777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.645170 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:02.645535 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.646176 (+ 641us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.646437 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 11:44:02.647733 22315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44260
I20260810 11:44:02.679445 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.651387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.20.93.190:34877)
0810 11:44:02.665380 (+ 13993us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.665386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.665407 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:44:02.665466 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.665469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.665532 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.665707 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:02.667159 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.668059 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.669356 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.669946 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.670090 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.672677 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:02.672683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:02.672687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:02.672728 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:02.676086 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.676833 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.676840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.676845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.677005 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.677438 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.677444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.677447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.677879 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:02.678143 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.678595 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.678896 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13864,"thread_start_us":13704,"threads_started":1}
W20260810 11:44:02.680137 22090 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43060
I20260810 11:44:02.683339 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22168
I20260810 11:44:02.698871 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.684546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44272 (local address 127.20.93.129:35823)
0810 11:44:02.684780 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.684791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.685266 (+ 475us) server_negotiation.cc:407] Connection header received
0810 11:44:02.685470 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.685477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.685565 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.685731 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:02.687194 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.688140 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.689459 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.690022 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.690170 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.692498 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:02.692509 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:02.692514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:02.692556 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:02.695550 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.696534 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.696540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.696543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.696709 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.697119 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.697125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.697129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.697490 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:02.697917 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.698182 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.698453 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 11:44:02.699482 22315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44272
W20260810 11:44:02.700089 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22168: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:02.700295 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22168
I20260810 11:44:02.742053 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22324
Aug 10 11:44:02 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:02.759021 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.742967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55640 (local address 127.20.93.130:44625)
0810 11:44:02.743573 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.743578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.743593 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:44:02.743648 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.743651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.743717 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.743838 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:02.745066 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.745987 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.747257 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.747728 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.747879 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.752841 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:02.752857 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:02.752874 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:02.752917 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:02.756025 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.756967 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.756975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.756978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.757112 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.757555 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.757558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.757560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.757963 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:02.758187 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.758569 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.758745 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":326,"threads_started":1}
W20260810 11:44:02.759868 22468 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55640
W20260810 11:44:02.760584 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22324: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:02.760792 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22324
I20260810 11:44:02.782470 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22479
Aug 10 11:44:02 dist-test-slave-njxd krb5kdc[22042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:02.799181 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:02.783423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.20.93.131:33205)
0810 11:44:02.784026 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:02.784031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:02.784047 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:44:02.784100 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:02.784103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:02.784163 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:02.784291 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:02.785632 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.786443 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.787678 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:02.788125 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:02.788273 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:02.793022 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:02.793039 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:02.793057 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:02.793100 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:02.796239 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.796947 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.796954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.796956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.797095 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:02.797486 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:02.797491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:02.797494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:02.797909 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:02.798239 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:02.798548 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 11:44:02.798799 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":317,"threads_started":1}
W20260810 11:44:02.799963 22617 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51534
W20260810 11:44:02.800729 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22479: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:02.800911 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22479
I20260810 11:44:02.829963 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22058
I20260810 11:44:02.831444 22133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:02.954599 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22058
2026-08-10T11:44:02Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5623 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:03 dist-test-slave-njxd krb5kdc[22729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:03 dist-test-slave-njxd krb5kdc[22729](info): set up 2 sockets
Aug 10 11:44:03 dist-test-slave-njxd krb5kdc[22729](info): commencing operation
krb5kdc: starting...
W20260810 11:44:04.126080 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:04 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:04.348646 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:37551
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:37551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:04.682510 22743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:04.682816 22743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:04.682989 22743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:04.690479 22743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:04.690582 22743 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:44:04.690671 22743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:04.690773 22743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:04.690865 22743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:04.690979 22743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:04.700933 22743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:37551
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:37551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:04.702379 22743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:04.703754 22743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:04.715621 22748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:04.715714 22749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:04.715730 22751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:04.716938 22743 server_base.cc:1034] running on GCE node
I20260810 11:44:04.717748 22743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:04.719197 22743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:04.720623 22743 hybrid_clock.cc:648] HybridClock initialized: now 1786362244720540 us; error 51 us; skew 500 ppm
I20260810 11:44:04.720999 22743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:04 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362244, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:04.724308 22743 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:04.725955 22743 webserver.cc:460] Webserver started at http://127.20.93.190:45185/ using document root <none> and password file <none>
I20260810 11:44:04.726711 22743 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:04.726850 22743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:04.727186 22743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:04.730301 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "299949918c2e421fa28a13a362014253"
format_stamp: "Formatted at 2026-08-10 11:44:04 on dist-test-slave-njxd"
server_key: "d3e6250de9e57aa59e99c59b204d597d"
server_key_iv: "fa1051d074561589fc7dc40057ec90eb"
server_key_version: "encryptionkey@0"
I20260810 11:44:04.731098 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "299949918c2e421fa28a13a362014253"
format_stamp: "Formatted at 2026-08-10 11:44:04 on dist-test-slave-njxd"
server_key: "d3e6250de9e57aa59e99c59b204d597d"
server_key_iv: "fa1051d074561589fc7dc40057ec90eb"
server_key_version: "encryptionkey@0"
I20260810 11:44:04.736519 22743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:44:04.740140 22760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:04.741477 22743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:04.741686 22743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "299949918c2e421fa28a13a362014253"
format_stamp: "Formatted at 2026-08-10 11:44:04 on dist-test-slave-njxd"
server_key: "d3e6250de9e57aa59e99c59b204d597d"
server_key_iv: "fa1051d074561589fc7dc40057ec90eb"
server_key_version: "encryptionkey@0"
I20260810 11:44:04.741940 22743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:04.760752 22743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:04.765717 22743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:04.766029 22743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:04.789057 22743 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:37551
I20260810 11:44:04.789090 22825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:37551 every 8 connection(s)
I20260810 11:44:04.790724 22743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:04.794714 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:04.799788 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22743
I20260810 11:44:04.800034 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:04.800722 20854 external_mini_cluster.cc:1428] Setting key f9cc0f27c3cf508fb4b3efb10a677357
I20260810 11:44:04.806321 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Bootstrap starting.
I20260810 11:44:04.809564 22826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:04.810719 22826 log.cc:824] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:04.814030 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: No bootstrap required, opened a new log
Aug 10 11:44:04 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:04.823706 22826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } }
I20260810 11:44:04.824198 22826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:04.824308 22826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299949918c2e421fa28a13a362014253, State: Initialized, Role: FOLLOWER
I20260810 11:44:04.825075 22826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } }
I20260810 11:44:04.825399 22826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:04.825558 22826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:04.825829 22826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:04.828616 22826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } }
I20260810 11:44:04.829231 22826 leader_election.cc:302] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299949918c2e421fa28a13a362014253; no voters:
I20260810 11:44:04.829923 22826 leader_election.cc:288] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:04.830114 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:04.802972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.20.93.190:37551)
0810 11:44:04.803891 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:04.803903 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:04.803955 (+ 52us) server_negotiation.cc:407] Connection header received
0810 11:44:04.805734 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:04.805759 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:04.806167 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:04.806577 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:04.808072 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:04.809393 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:04.810868 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:04.811947 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:04.812169 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:04.821243 (+ 9074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:04.821255 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:04.821278 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:04.821347 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:04.825389 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:04.826282 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:04.826293 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:04.826304 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:04.826541 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:04.826947 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:04.826952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:04.826957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:04.827566 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:04.827908 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:04.828353 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 11:44:04.828840 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":422,"threads_started":1}
I20260810 11:44:04.830085 22831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:04.832571 22826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:04.835175 22831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [term 1 LEADER]: Becoming Leader. State: Replica: 299949918c2e421fa28a13a362014253, State: Running, Role: LEADER
I20260810 11:44:04.835940 22831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } }
I20260810 11:44:04.839584 22832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "299949918c2e421fa28a13a362014253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } } }
I20260810 11:44:04.839985 22832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:04.840562 22834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299949918c2e421fa28a13a362014253. Latest consensus state: current_term: 1 leader_uuid: "299949918c2e421fa28a13a362014253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299949918c2e421fa28a13a362014253" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 37551 } } }
I20260810 11:44:04.842280 22834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253 [sys.catalog]: This master's current role is: LEADER
W20260810 11:44:04.845940 22846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:44:04.846170 22846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:44:04.846385 22847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:04.849370 22847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:04.859345 22847 catalog_manager.cc:1347] Generated new cluster ID: 2b97d6439183449e93fe7fc46303c98f
I20260810 11:44:04.859464 22847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:04.877935 22847 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:04.879468 22847 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:04.890234 22847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Generated new TSK 0
I20260810 11:44:04.891381 22847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:04.952889 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:37551
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:05.294883 22855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:05.295178 22855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:05.295363 22855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:05.302753 22855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:05.302877 22855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:05.303050 22855 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:44:05.303133 22855 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:05.303265 22855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:05.313429 22855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:37551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:05.314982 22855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:05.316360 22855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:05.329679 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:05.329710 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:05.329975 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:05.330511 22855 server_base.cc:1034] running on GCE node
I20260810 11:44:05.331154 22855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:05.332163 22855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:05.333376 22855 hybrid_clock.cc:648] HybridClock initialized: now 1786362245333314 us; error 53 us; skew 500 ppm
I20260810 11:44:05.333727 22855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:05 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362245, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:05.337015 22855 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:05.338615 22855 webserver.cc:460] Webserver started at http://127.20.93.129:38447/ using document root <none> and password file <none>
I20260810 11:44:05.339318 22855 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:05.339468 22855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:05.339795 22855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:05.342844 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "14267d30613e4e8189c411debda5acb8"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "25e58ae4d8c632e029bcda5364283c8e"
server_key_iv: "0cbdbd5b9606aa8be7534e9ac2dd76d0"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.343622 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "14267d30613e4e8189c411debda5acb8"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "25e58ae4d8c632e029bcda5364283c8e"
server_key_iv: "0cbdbd5b9606aa8be7534e9ac2dd76d0"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.349098 22855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:44:05.352779 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:05.354228 22855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:44:05.354425 22855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "14267d30613e4e8189c411debda5acb8"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "25e58ae4d8c632e029bcda5364283c8e"
server_key_iv: "0cbdbd5b9606aa8be7534e9ac2dd76d0"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.354662 22855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:05.381629 22855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:05.385962 22855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:05.386310 22855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:05.389307 22855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:05.389449 22855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:05.389628 22855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:05.389770 22855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 11:44:05 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362245, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:05.416807 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.389679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:51213 (local address 127.20.93.190:37551)
0810 11:44:05.390264 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:05.390281 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:05.391748 (+ 1467us) server_negotiation.cc:407] Connection header received
0810 11:44:05.394964 (+ 3216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:05.394970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:05.395062 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:05.395176 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:05.397725 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.399085 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.401647 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.402365 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.402526 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.408188 (+ 5662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:05.408201 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:05.408207 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:05.408258 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:05.412403 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.413623 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.413630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.413634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.413781 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.414441 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.414447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.414450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.414889 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:05.415047 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:05.416147 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0810 11:44:05.416413 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":281,"threads_started":1}
I20260810 11:44:05.418115 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.390510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.129:51213)
0810 11:44:05.391546 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:05.391594 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:05.394540 (+ 2946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:05.395617 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:05.395634 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:05.396524 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:05.397447 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.397483 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.399305 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.399314 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.401456 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.401471 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.402881 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.402888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.403195 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.404122 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:05.404161 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:05.407904 (+ 3743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:05.412664 (+ 4760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.412680 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.412705 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.413385 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.413960 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.413967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.413972 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.414306 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.415039 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:05.415049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:05.415650 (+ 601us) client_negotiation.cc:769] Sending connection context
0810 11:44:05.416041 (+ 391us) client_negotiation.cc:240] Negotiation successful
0810 11:44:05.416620 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":344,"threads_started":1}
I20260810 11:44:05.441087 22855 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:36543
I20260810 11:44:05.441116 23004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:36543 every 8 connection(s)
I20260810 11:44:05.442932 22855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:05.443239 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.425687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:45257 (local address 127.20.93.190:37551)
0810 11:44:05.425912 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:05.425920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:05.426961 (+ 1041us) server_negotiation.cc:407] Connection header received
0810 11:44:05.427202 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:05.427210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:05.427304 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:05.427425 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:05.429251 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.430066 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.432592 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.433403 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.433551 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.436959 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:05.436966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:05.436972 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:05.437021 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:05.439987 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.441122 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.441129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.441133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.441317 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.441748 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.441751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.441753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.442039 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:05.442162 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:05.442704 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 11:44:05.442906 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:44:05.443600 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.425665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.129:45257)
0810 11:44:05.426814 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:05.426839 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:05.427042 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:05.427492 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:05.427503 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:05.428458 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:05.429070 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.429091 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.430324 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.430333 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.432366 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.432393 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.433720 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.433730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.434170 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.435074 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:05.435107 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:05.436665 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:05.440149 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.440157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.440169 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.440975 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.441439 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.441444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.441447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.441628 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.442167 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:05.442175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:05.442408 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 11:44:05.442582 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 11:44:05.442974 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":722,"threads_started":1}
I20260810 11:44:05.445847 23005 heartbeater.cc:347] Connected to a master server at 127.20.93.190:37551
I20260810 11:44:05.446131 23005 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:05.446815 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 22855
I20260810 11:44:05.447045 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:05.447721 20854 external_mini_cluster.cc:1428] Setting key 0fcfa0cef2ec18ca0396f0794e0216a4
I20260810 11:44:05.448992 23005 heartbeater.cc:511] Master 127.20.93.190:37551 requested a full tablet report, sending...
I20260810 11:44:05.451479 22779 ts_manager.cc:194] Registered new tserver with Master: 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543)
I20260810 11:44:05.453725 22779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:51213
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:05.497188 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:37551
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:05.832240 23010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:05.832576 23010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:05.832770 23010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:05.840358 23010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:05.840516 23010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:05.840683 23010 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:44:05.840770 23010 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:05.840910 23010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:05.853065 23010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:37551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:05.854503 23010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:05.855895 23010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:05.869408 23017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:05.869446 23019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:05.869469 23016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:05.870296 23010 server_base.cc:1034] running on GCE node
I20260810 11:44:05.870908 23010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:05.871850 23010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:05.873075 23010 hybrid_clock.cc:648] HybridClock initialized: now 1786362245873024 us; error 63 us; skew 500 ppm
I20260810 11:44:05.873394 23010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:05 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362245, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:05.876663 23010 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:05.878177 23010 webserver.cc:460] Webserver started at http://127.20.93.130:45325/ using document root <none> and password file <none>
I20260810 11:44:05.878913 23010 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:05.879060 23010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:05.879375 23010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:05.882303 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "f11eb6294528495dbe14c75ee0ab8368"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "ae3e64b675432e45ef15884745711dc8"
server_key_iv: "f35b114723d9fdfae334e47e2e55dc8a"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.883075 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "f11eb6294528495dbe14c75ee0ab8368"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "ae3e64b675432e45ef15884745711dc8"
server_key_iv: "f35b114723d9fdfae334e47e2e55dc8a"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.888427 23010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:44:05.892314 23026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:05.893767 23010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:05.893932 23010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "f11eb6294528495dbe14c75ee0ab8368"
format_stamp: "Formatted at 2026-08-10 11:44:05 on dist-test-slave-njxd"
server_key: "ae3e64b675432e45ef15884745711dc8"
server_key_iv: "f35b114723d9fdfae334e47e2e55dc8a"
server_key_version: "encryptionkey@0"
I20260810 11:44:05.894147 23010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:05.921921 23010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:05.926486 23010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:05.926780 23010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:05.929344 23010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:05.929466 23010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:05.929641 23010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:05.929756 23010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:05 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362245, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:05.954578 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.930174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:33413 (local address 127.20.93.190:37551)
0810 11:44:05.930388 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:05.930395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:05.931754 (+ 1359us) server_negotiation.cc:407] Connection header received
0810 11:44:05.934108 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:05.934117 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:05.934224 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:05.934345 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:05.937436 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.938436 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.940911 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.941540 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.941706 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.946164 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:05.946169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:05.946172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:05.946205 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:05.949413 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.950249 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.950254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.950259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.950436 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.951088 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.951096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.951102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.951539 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:05.951747 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:05.953837 (+ 2090us) server_negotiation.cc:299] Negotiation successful
0810 11:44:05.954115 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:44:05.955681 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.930688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.130:33413)
0810 11:44:05.931570 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:05.931622 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:05.933795 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:05.935014 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:05.935026 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:05.936032 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:05.937183 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.937219 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.938623 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.938628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.940757 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.940769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.941827 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.941838 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.942059 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.942920 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:05.942942 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:05.945905 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:05.949622 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.949634 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.949658 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.950098 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.950601 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.950606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.950611 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.950898 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.951848 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:05.951863 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:05.952458 (+ 595us) client_negotiation.cc:769] Sending connection context
0810 11:44:05.952877 (+ 419us) client_negotiation.cc:240] Negotiation successful
0810 11:44:05.953652 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":387,"threads_started":1}
I20260810 11:44:05.978156 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.963848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:50653 (local address 127.20.93.190:37551)
0810 11:44:05.964016 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:05.964020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:05.964754 (+ 734us) server_negotiation.cc:407] Connection header received
0810 11:44:05.964941 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:05.964946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:05.965003 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:05.965103 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:05.966367 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.967716 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.969366 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.969836 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.969976 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.971771 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:05.971776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:05.971779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:05.971809 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:05.974706 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.975713 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.975722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.975728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.975913 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:05.976386 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:05.976393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:05.976396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:05.976895 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:05.977051 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:05.977504 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 11:44:05.977764 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 11:44:05.978397 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:05.963983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.130:50653)
0810 11:44:05.964643 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:05.964658 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:05.964816 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:05.965131 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:05.965135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:05.965675 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:05.966174 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.966197 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.967876 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.967880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.969202 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:05.969216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:05.970036 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:05.970045 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:05.970205 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:05.970833 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:05.970849 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:05.971547 (+ 698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:05.974848 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.974854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.974858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.975524 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.976074 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:05.976084 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:05.976090 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:05.976267 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:05.977021 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:05.977028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:05.977226 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 11:44:05.977412 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 11:44:05.977818 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":382,"threads_started":1}
I20260810 11:44:05.981419 23010 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:46247
I20260810 11:44:05.981455 23157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:46247 every 8 connection(s)
I20260810 11:44:05.983050 23010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:05.984980 23158 heartbeater.cc:347] Connected to a master server at 127.20.93.190:37551
I20260810 11:44:05.985219 23158 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:05.985888 23158 heartbeater.cc:511] Master 127.20.93.190:37551 requested a full tablet report, sending...
I20260810 11:44:05.987121 22779 ts_manager.cc:194] Registered new tserver with Master: f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247)
I20260810 11:44:05.987913 22779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:33413
I20260810 11:44:05.991707 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23010
I20260810 11:44:05.991890 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:05.992341 20854 external_mini_cluster.cc:1428] Setting key 84144e9c5f69046fc53fa26d6f5b37e2
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:06.040128 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:37551
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:06.376037 23162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:06.376335 23162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:06.376513 23162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:06.383850 23162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:06.383965 23162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:06.384148 23162 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:44:06.384228 23162 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:06.384357 23162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:06.394671 23162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:37551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:06.396130 23162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:06.397634 23162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:06.411441 23168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:06.411424 23169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:06.411445 23171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:06.412209 23162 server_base.cc:1034] running on GCE node
I20260810 11:44:06.412886 23162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:06.413954 23162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:06.415185 23162 hybrid_clock.cc:648] HybridClock initialized: now 1786362246415146 us; error 55 us; skew 500 ppm
I20260810 11:44:06.415534 23162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:06 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362246, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:06.418829 23162 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:06.420331 23162 webserver.cc:460] Webserver started at http://127.20.93.131:38669/ using document root <none> and password file <none>
I20260810 11:44:06.421106 23162 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:06.421264 23162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:06.421644 23162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:06.424614 23162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "ecb33c2170464b5e95db929574b924dd"
format_stamp: "Formatted at 2026-08-10 11:44:06 on dist-test-slave-njxd"
server_key: "b0f78c4dfec1a1f6ab675e2712bd9b01"
server_key_iv: "c5d8518447cf586f360fe0eec6753bc8"
server_key_version: "encryptionkey@0"
I20260810 11:44:06.425395 23162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "ecb33c2170464b5e95db929574b924dd"
format_stamp: "Formatted at 2026-08-10 11:44:06 on dist-test-slave-njxd"
server_key: "b0f78c4dfec1a1f6ab675e2712bd9b01"
server_key_iv: "c5d8518447cf586f360fe0eec6753bc8"
server_key_version: "encryptionkey@0"
I20260810 11:44:06.430924 23162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 11:44:06.434648 23178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:06.435997 23162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 11:44:06.436177 23162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "ecb33c2170464b5e95db929574b924dd"
format_stamp: "Formatted at 2026-08-10 11:44:06 on dist-test-slave-njxd"
server_key: "b0f78c4dfec1a1f6ab675e2712bd9b01"
server_key_iv: "c5d8518447cf586f360fe0eec6753bc8"
server_key_version: "encryptionkey@0"
I20260810 11:44:06.436393 23162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:06.456882 23005 heartbeater.cc:503] Master 127.20.93.190:37551 was elected leader, sending a full tablet report...
I20260810 11:44:06.466288 23162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:06.470981 23162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:06.471297 23162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:06.474068 23162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:06.474210 23162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:06.474330 23162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:06.474423 23162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:06 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362246, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:06.500586 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.474836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:47789 (local address 127.20.93.190:37551)
0810 11:44:06.475067 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.475077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.476822 (+ 1745us) server_negotiation.cc:407] Connection header received
0810 11:44:06.479319 (+ 2497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.479325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.479426 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.479553 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:06.482743 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.483836 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.486364 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.487182 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.487352 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.492769 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:06.492774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:06.492777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:06.492819 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:06.496555 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.497726 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.497734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.497738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.497945 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.498539 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.498547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.498553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.498919 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:06.499101 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.499943 (+ 842us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.500175 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 11:44:06.501855 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.475775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.131:47789)
0810 11:44:06.476647 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.476688 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.478999 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.479925 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.479953 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.481206 (+ 1253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:06.482460 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.482498 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.484018 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.484022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.486150 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.486175 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.487433 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.487439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.487636 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.488490 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:06.488514 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:06.492442 (+ 3928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:06.496752 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.496764 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.496792 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.497499 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.498103 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.498107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.498109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.498345 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.499077 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:06.499085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:06.499461 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.499757 (+ 296us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.500332 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":367,"threads_started":1}
I20260810 11:44:06.523185 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.508710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.20.93.131:43169)
0810 11:44:06.509339 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.509355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.509516 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.509960 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.509965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.510733 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:06.511172 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.511184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.512177 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.512182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.513210 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.513222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.514208 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.514216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.514409 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.515115 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:06.515133 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:06.515934 (+ 801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:06.519867 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.519873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.519876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.520540 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.520933 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.520938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.520941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.521125 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.522063 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:06.522068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:06.522336 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.522538 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.522868 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":350,"threads_started":1}
I20260810 11:44:06.523571 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.508563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:43169 (local address 127.20.93.190:37551)
0810 11:44:06.508739 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.508744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.509449 (+ 705us) server_negotiation.cc:407] Connection header received
0810 11:44:06.509724 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.509730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.509818 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.509927 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:06.511320 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.512029 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.513365 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.514026 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.514161 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.516140 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:06.516154 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:06.516158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:06.516192 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:06.519604 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.520668 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.520672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.520675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.520806 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.521491 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.521494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.521496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.521948 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:06.522108 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.522726 (+ 618us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.523055 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 11:44:06.531503 23162 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:37639
I20260810 11:44:06.531587 23306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:37639 every 8 connection(s)
I20260810 11:44:06.532878 23162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:06.535024 23307 heartbeater.cc:347] Connected to a master server at 127.20.93.190:37551
I20260810 11:44:06.535260 23307 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:06.535923 23307 heartbeater.cc:511] Master 127.20.93.190:37551 requested a full tablet report, sending...
I20260810 11:44:06.537264 22779 ts_manager.cc:194] Registered new tserver with Master: ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:06.538141 22779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:47789
I20260810 11:44:06.541877 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23162
I20260810 11:44:06.542054 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:06.542505 20854 external_mini_cluster.cc:1428] Setting key 9adda667d4eb8bdc814d740d3897b12b
I20260810 11:44:06.545500 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:44:06 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:44:06 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:06.597041 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.583669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.20.93.190:37551)
0810 11:44:06.583924 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.583932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.584133 (+ 201us) server_negotiation.cc:407] Connection header received
0810 11:44:06.584294 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.584298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.584364 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.584454 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:06.585954 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.586713 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.587872 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.588290 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.588389 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.592153 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:06.592158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:06.592161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:06.592195 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:06.594647 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.595386 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.595390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.595393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.595520 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.595863 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.595866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.595868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.596143 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:06.596337 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.596585 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.596770 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 11:44:06.601341 22779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:06.608565 22779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:06.650689 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.621271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.0.0.1:53938)
0810 11:44:06.628470 (+ 7199us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.628525 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.628772 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.633604 (+ 4832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.633612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.633647 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:06.634221 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.634254 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.636569 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.636575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.639344 (+ 2769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.639360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.649076 (+ 9716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.649083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.649370 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.649854 (+ 484us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.650009 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.650218 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6831,"mutex_wait_us":393,"thread_start_us":599,"threads_started":1}
I20260810 11:44:06.651489 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.621497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.20.93.129:36543)
0810 11:44:06.632755 (+ 11258us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.632769 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.632812 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:44:06.632958 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.632970 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.633315 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.633594 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:06.634546 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.636387 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.639605 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.645767 (+ 6162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.645933 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.646209 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.650985 (+ 4776us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.651182 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11010,"thread_start_us":12300,"threads_started":1}
I20260810 11:44:06.655263 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.622866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46247 (local address 127.0.0.1:40734)
0810 11:44:06.637948 (+ 15082us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.637966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.638097 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.638969 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.638974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.638996 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:06.639501 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.639516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.641745 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.641753 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.651448 (+ 9695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.651464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.653972 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.653981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.654253 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.654315 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.654433 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.654661 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14972,"mutex_wait_us":96,"thread_start_us":14091,"threads_started":1}
I20260810 11:44:06.656258 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.623249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.20.93.130:46247)
0810 11:44:06.636707 (+ 13458us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.636717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.638225 (+ 1508us) server_negotiation.cc:407] Connection header received
0810 11:44:06.638328 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.638335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.638574 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.638770 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:06.639715 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.641524 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.651654 (+ 10130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.653329 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.653494 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.653793 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.655636 (+ 1843us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.655873 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13280,"thread_start_us":13154,"threads_started":1}
I20260810 11:44:06.655977 22927 tablet_service.cc:1467] Processing CreateTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (DEFAULT_TABLE table=test-table [id=c5abf78940d2436891dc02b0078c7dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:06.657079 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.622213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.20.93.131:37639)
0810 11:44:06.636051 (+ 13838us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.636061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.637447 (+ 1386us) server_negotiation.cc:407] Connection header received
0810 11:44:06.638162 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.638171 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.638472 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.638683 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:06.639941 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.644993 (+ 5052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.653597 (+ 8604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.655082 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.655213 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.655461 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.656580 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.656759 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13649,"thread_start_us":13810,"threads_started":1}
I20260810 11:44:06.656448 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.621519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:37639 (local address 127.0.0.1:59390)
0810 11:44:06.637223 (+ 15704us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.637245 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.637930 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.639023 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.639030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.639068 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:06.639635 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.639649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.647973 (+ 8324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.647981 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.653329 (+ 5348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.653344 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.655617 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.655624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.655806 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.655836 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.655927 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.656069 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15554,"mutex_wait_us":1069,"thread_start_us":16126,"threads_started":1}
I20260810 11:44:06.657563 22927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.660727 23081 tablet_service.cc:1467] Processing CreateTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (DEFAULT_TABLE table=test-table [id=c5abf78940d2436891dc02b0078c7dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:06.661096 23231 tablet_service.cc:1467] Processing CreateTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (DEFAULT_TABLE table=test-table [id=c5abf78940d2436891dc02b0078c7dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:06.662483 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.662599 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.679728 23329 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Bootstrap starting.
I20260810 11:44:06.684410 23331 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Bootstrap starting.
I20260810 11:44:06.684904 23329 tablet_bootstrap.cc:654] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.686411 23329 log.cc:824] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:06.688482 23331 tablet_bootstrap.cc:654] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.690196 23331 log.cc:824] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:06.692216 23330 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Bootstrap starting.
I20260810 11:44:06.695798 23330 tablet_bootstrap.cc:654] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.697343 23330 log.cc:824] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:06.705170 23329 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: No bootstrap required, opened a new log
I20260810 11:44:06.705597 23329 ts_tablet_manager.cc:1397] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.008s
I20260810 11:44:06.716979 23331 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: No bootstrap required, opened a new log
I20260810 11:44:06.717027 23330 tablet_bootstrap.cc:492] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: No bootstrap required, opened a new log
I20260810 11:44:06.717293 23331 ts_tablet_manager.cc:1397] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.022s
I20260810 11:44:06.717321 23330 ts_tablet_manager.cc:1397] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.005s
I20260810 11:44:06.719823 23329 raft_consensus.cc:348] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.720427 23329 raft_consensus.cc:374] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.720585 23329 raft_consensus.cc:729] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.721406 23329 consensus_queue.cc:260] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.729912 23329 ts_tablet_manager.cc:1428] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Time spent starting tablet: real 0.024s user 0.018s sys 0.004s
I20260810 11:44:06.730005 23331 raft_consensus.cc:348] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.730481 23331 raft_consensus.cc:374] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.730607 23331 raft_consensus.cc:729] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.731262 23331 consensus_queue.cc:260] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.733225 23330 raft_consensus.cc:348] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.733819 23330 raft_consensus.cc:374] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.733996 23330 raft_consensus.cc:729] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.734915 23330 consensus_queue.cc:260] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.739348 23307 heartbeater.cc:503] Master 127.20.93.190:37551 was elected leader, sending a full tablet report...
I20260810 11:44:06.740120 23331 ts_tablet_manager.cc:1428] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Time spent starting tablet: real 0.023s user 0.008s sys 0.016s
I20260810 11:44:06.741554 23158 heartbeater.cc:503] Master 127.20.93.190:37551 was elected leader, sending a full tablet report...
I20260810 11:44:06.742978 23330 ts_tablet_manager.cc:1428] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Time spent starting tablet: real 0.026s user 0.020s sys 0.003s
W20260810 11:44:06.784668 23308 tablet.cc:2369] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:06.837399 23336 raft_consensus.cc:482] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:06.837766 23336 raft_consensus.cc:504] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.839556 23336 leader_election.cc:288] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247)
I20260810 11:44:06.848551 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.840008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:39807 (local address 127.20.93.129:36543)
0810 11:44:06.840201 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.840206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.840525 (+ 319us) server_negotiation.cc:407] Connection header received
0810 11:44:06.840752 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.840758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.840863 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.840955 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:06.841785 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.843151 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.846055 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.847285 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.847430 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.847500 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.848050 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.848293 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:44:06.848703 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.840055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.20.93.131:39807)
0810 11:44:06.840394 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.840428 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.840587 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.841287 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.841294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.841327 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:06.841557 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.841593 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.843353 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.843361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.845862 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.845882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.847638 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.847645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.847849 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.847887 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.847991 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.848133 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3}
I20260810 11:44:06.850010 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.840970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:38127 (local address 127.20.93.130:46247)
0810 11:44:06.841184 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.841190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.841741 (+ 551us) server_negotiation.cc:407] Connection header received
0810 11:44:06.841985 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.841990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.842062 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.842168 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:06.842946 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.843904 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.845919 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.847242 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.847424 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.847500 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.849456 (+ 1956us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.849675 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:44:06.850865 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.840577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46247 (local address 127.20.93.131:38127)
0810 11:44:06.841635 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.841653 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.841838 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.842338 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.842343 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.842369 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:06.842750 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.842766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.844061 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.844064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.845757 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.845770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.847477 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.847486 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.849217 (+ 1731us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.849271 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.850206 (+ 935us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.850389 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":3,"thread_start_us":631,"threads_started":1}
I20260810 11:44:06.850888 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5959e7a4fbab4bc8ae76e1d549c8c9d8" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8" is_pre_election: true
I20260810 11:44:06.851574 22948 raft_consensus.cc:2454] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 0.
I20260810 11:44:06.852319 23180 leader_election.cc:302] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, ecb33c2170464b5e95db929574b924dd; no voters:
I20260810 11:44:06.852803 23336 raft_consensus.cc:2781] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:06.852996 23336 raft_consensus.cc:482] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:06.853139 23336 raft_consensus.cc:3037] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:06.852916 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5959e7a4fbab4bc8ae76e1d549c8c9d8" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368" is_pre_election: true
I20260810 11:44:06.853461 23103 raft_consensus.cc:2454] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 0.
I20260810 11:44:06.857033 23336 raft_consensus.cc:504] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.857946 23336 leader_election.cc:288] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 election: Requested vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247)
I20260810 11:44:06.858394 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5959e7a4fbab4bc8ae76e1d549c8c9d8" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368"
I20260810 11:44:06.858392 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5959e7a4fbab4bc8ae76e1d549c8c9d8" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8"
I20260810 11:44:06.858639 23103 raft_consensus.cc:3037] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:06.858654 22948 raft_consensus.cc:3037] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:06.861059 23103 raft_consensus.cc:2454] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 1.
I20260810 11:44:06.861469 23180 leader_election.cc:302] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecb33c2170464b5e95db929574b924dd, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:06.861763 23336 raft_consensus.cc:2781] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:06.862316 23336 raft_consensus.cc:686] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 1 LEADER]: Becoming Leader. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Running, Role: LEADER
I20260810 11:44:06.862562 22948 raft_consensus.cc:2454] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 1.
I20260810 11:44:06.863349 23336 consensus_queue.cc:237] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } }
I20260810 11:44:06.868695 22777 catalog_manager.cc:5495] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd reported cstate change: term changed from 0 to 1, leader changed from <none> to ecb33c2170464b5e95db929574b924dd (127.20.93.131). New cstate: current_term: 1 leader_uuid: "ecb33c2170464b5e95db929574b924dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } health_report { overall_health: UNKNOWN } } }
Aug 10 11:44:06 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362244, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:06.939808 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.924746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.20.93.190:37551)
0810 11:44:06.924921 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:06.924926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:06.925693 (+ 767us) server_negotiation.cc:407] Connection header received
0810 11:44:06.925912 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:06.925917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:06.925995 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:06.926100 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:06.927526 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.928321 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.929865 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.930303 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.930447 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.934175 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:06.934180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:06.934183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:06.934211 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:06.936942 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.937715 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.937720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.937722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.937834 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:06.938243 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:06.938251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:06.938255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:06.938529 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:06.938683 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:06.939136 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 11:44:06.939481 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":56}
I20260810 11:44:06.939848 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:06.924881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:37551 (local address 127.0.0.1:43464)
0810 11:44:06.925524 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:06.925541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:06.925783 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:06.926124 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:06.926129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:06.926969 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:06.927370 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.927391 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.928460 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.928468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.929498 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:06.929509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:06.930494 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:06.930499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:06.930664 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:06.931390 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:06.931408 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:06.933947 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:06.937069 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.937085 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.937094 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.937569 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.937940 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:06.937947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:06.937950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:06.938116 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:06.938658 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:06.938666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:06.938858 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 11:44:06.939025 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 11:44:06.939335 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":372,"threads_started":1}
W20260810 11:44:06.946012 23006 tablet.cc:2369] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:06.946416 22777 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43464:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:44:06.948078 22777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:06.956017 22927 tablet_service.cc:1467] Processing CreateTablet for tablet 9591a3aa803e445aab52c0374dffee82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb02f05cabee4e098457391df1def472]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:06.956890 22927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9591a3aa803e445aab52c0374dffee82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.959043 23231 tablet_service.cc:1467] Processing CreateTablet for tablet 9591a3aa803e445aab52c0374dffee82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb02f05cabee4e098457391df1def472]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:06.959245 23081 tablet_service.cc:1467] Processing CreateTablet for tablet 9591a3aa803e445aab52c0374dffee82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb02f05cabee4e098457391df1def472]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:06.960136 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9591a3aa803e445aab52c0374dffee82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.960407 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9591a3aa803e445aab52c0374dffee82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:06.964416 23329 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: Bootstrap starting.
I20260810 11:44:06.965847 23329 tablet_bootstrap.cc:654] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.967891 23329 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: No bootstrap required, opened a new log
I20260810 11:44:06.968070 23329 ts_tablet_manager.cc:1397] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:06.968703 23329 raft_consensus.cc:348] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:06.968924 23329 raft_consensus.cc:374] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.969015 23329 raft_consensus.cc:729] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.969291 23329 consensus_queue.cc:260] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:06.969681 23330 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: Bootstrap starting.
I20260810 11:44:06.969784 23335 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:06.970151 23335 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:06.971457 23330 tablet_bootstrap.cc:654] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.971937 23329 ts_tablet_manager.cc:1428] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:06.976123 23331 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: Bootstrap starting.
I20260810 11:44:06.977115 23330 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: No bootstrap required, opened a new log
I20260810 11:44:06.977279 23330 ts_tablet_manager.cc:1397] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 11:44:06.977986 23331 tablet_bootstrap.cc:654] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:06.978138 23330 raft_consensus.cc:348] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:06.978427 23330 raft_consensus.cc:374] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.978524 23330 raft_consensus.cc:729] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.978791 23330 consensus_queue.cc:260] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:06.979331 23337 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:06.979730 23337 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:06.980261 23330 ts_tablet_manager.cc:1428] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:06.982395 23331 tablet_bootstrap.cc:492] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: No bootstrap required, opened a new log
I20260810 11:44:06.982556 23331 ts_tablet_manager.cc:1397] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260810 11:44:06.984122 23331 raft_consensus.cc:348] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:06.984370 23331 raft_consensus.cc:374] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:06.984464 23331 raft_consensus.cc:729] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Initialized, Role: FOLLOWER
I20260810 11:44:06.984876 23331 consensus_queue.cc:260] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
W20260810 11:44:06.985138 23159 tablet.cc:2369] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:06.985292 23336 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:06.985505 23331 ts_tablet_manager.cc:1428] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 11:44:06.985666 23336 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:07.181318 23335 raft_consensus.cc:482] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:07.181648 23335 raft_consensus.cc:504] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.183314 23335 leader_election.cc:288] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:07.190637 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.184275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:57099 (local address 127.20.93.130:46247)
0810 11:44:07.184523 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.184531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.185224 (+ 693us) server_negotiation.cc:407] Connection header received
0810 11:44:07.185413 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.185417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.185478 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.185583 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:07.186222 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.186985 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.188840 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.189650 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.189853 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.189935 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.190219 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.190348 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:44:07.190728 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.183798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:37431 (local address 127.20.93.131:37639)
0810 11:44:07.184407 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.184417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.185119 (+ 702us) server_negotiation.cc:407] Connection header received
0810 11:44:07.185364 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.185369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.185430 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.185542 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:07.186227 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.186945 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.188865 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.189707 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.189889 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.189972 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.190275 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.190418 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":745,"threads_started":1}
I20260810 11:44:07.190801 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.183824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46247 (local address 127.20.93.129:57099)
0810 11:44:07.185119 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.185133 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.185252 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.185620 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.185625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.185648 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:07.186043 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.186052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.187125 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.187130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.188677 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.188693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.189896 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.189903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.190059 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.190083 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.190192 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.190335 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":20,"thread_start_us":980,"threads_started":1}
I20260810 11:44:07.191207 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.183798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:37639 (local address 127.20.93.129:37431)
0810 11:44:07.184977 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.185001 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.185215 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.185561 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.185565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.185618 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:07.186028 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.186039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.187086 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.187093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.188704 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.188717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.189942 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.189949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.190099 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.190121 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.190246 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.190369 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"mutex_wait_us":49,"thread_start_us":815,"threads_started":1}
I20260810 11:44:07.191701 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9591a3aa803e445aab52c0374dffee82" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368" is_pre_election: true
I20260810 11:44:07.191999 23103 raft_consensus.cc:2454] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 0.
I20260810 11:44:07.192202 23254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9591a3aa803e445aab52c0374dffee82" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd" is_pre_election: true
I20260810 11:44:07.192533 22872 leader_election.cc:302] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:07.192586 23254 raft_consensus.cc:2454] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 0.
I20260810 11:44:07.192942 23335 raft_consensus.cc:2781] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:07.193090 23335 raft_consensus.cc:482] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:07.193207 23335 raft_consensus.cc:3037] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.195528 23335 raft_consensus.cc:504] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.196198 23335 leader_election.cc:288] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 election: Requested vote from peers f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:07.196736 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9591a3aa803e445aab52c0374dffee82" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368"
I20260810 11:44:07.196760 23254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9591a3aa803e445aab52c0374dffee82" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd"
I20260810 11:44:07.196950 23103 raft_consensus.cc:3037] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.196977 23254 raft_consensus.cc:3037] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.199684 23103 raft_consensus.cc:2454] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 1.
I20260810 11:44:07.200021 23254 raft_consensus.cc:2454] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 1.
I20260810 11:44:07.200098 22872 leader_election.cc:302] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:07.200529 23335 raft_consensus.cc:2781] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:07.200927 23335 raft_consensus.cc:686] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [term 1 LEADER]: Becoming Leader. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Running, Role: LEADER
I20260810 11:44:07.201442 23335 consensus_queue.cc:237] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.204635 23355 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: TxnStatusTablet state changed. Reason: New leader 14267d30613e4e8189c411debda5acb8. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.205075 23355 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:07.205912 23357 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:44:07.206179 23357 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:44:07.206825 22777 catalog_manager.cc:5495] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14267d30613e4e8189c411debda5acb8 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:07.227111 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.220815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.0.0.1:53950)
0810 11:44:07.221081 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.221098 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.221205 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.221791 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.221794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.221805 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:07.221963 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.221970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.223430 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.223433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.224411 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.224423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.225393 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.225397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.225541 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.225655 (+ 114us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.226691 (+ 1036us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.226707 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.226759 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.226891 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 11:44:07.227237 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.220935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.20.93.129:36543)
0810 11:44:07.221195 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.221201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.221230 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:44:07.221458 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.221463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.221598 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.221761 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.222091 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.223303 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.224570 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.225237 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.225368 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.225851 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.226580 (+ 729us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.226749 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.226808 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.226896 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260810 11:44:07.235353 23103 raft_consensus.cc:1261] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Refusing update from remote peer 14267d30613e4e8189c411debda5acb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:07.235481 23254 raft_consensus.cc:1261] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Refusing update from remote peer 14267d30613e4e8189c411debda5acb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:07.236395 23355 consensus_queue.cc:1035] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.236768 23335 consensus_queue.cc:1035] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.251039 23337 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: TxnStatusTablet state changed. Reason: New leader 14267d30613e4e8189c411debda5acb8. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.251361 23337 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:07.251384 23336 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: TxnStatusTablet state changed. Reason: New leader 14267d30613e4e8189c411debda5acb8. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.251668 23336 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:07.255527 23360 mvcc.cc:205] Tried to move back new op lower bound from 7316939764665171968 to 7316939764541276160. Current Snapshot: MvccSnapshot[applied={T|T < 7316939764665171968}]
I20260810 11:44:07.257911 23362 mvcc.cc:205] Tried to move back new op lower bound from 7316939764665171968 to 7316939764541276160. Current Snapshot: MvccSnapshot[applied={T|T < 7316939764665171968}]
I20260810 11:44:07.263809 23337 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.264072 23337 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P f11eb6294528495dbe14c75ee0ab8368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:07.266564 23336 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.266886 23336 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P ecb33c2170464b5e95db929574b924dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:07.269294 23365 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.269765 23365 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:07.270998 23336 consensus_queue.cc:1035] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.271795 23335 tablet_replica.cc:439] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } } }
I20260810 11:44:07.272073 23335 tablet_replica.cc:442] T 9591a3aa803e445aab52c0374dffee82 P 14267d30613e4e8189c411debda5acb8: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:07.280200 23336 consensus_queue.cc:1035] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.285976 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.278336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.20.93.131:37639)
0810 11:44:07.278598 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.278605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.278967 (+ 362us) server_negotiation.cc:407] Connection header received
0810 11:44:07.279304 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.279310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.279409 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.279540 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.280327 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.281309 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.282963 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.283607 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.283735 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.284429 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.285167 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.285306 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.285542 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.285645 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260810 11:44:07.303954 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.297337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.20.93.131:40861)
0810 11:44:07.298137 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.298158 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.298360 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.298796 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.298799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.298808 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:07.299083 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.299090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.300531 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.300535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.301708 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.301721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.302794 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.302799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.302913 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.302966 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.303409 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.303417 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.303478 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.303578 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":460,"threads_started":1}
I20260810 11:44:07.303926 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.297341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:40861 (local address 127.20.93.129:36543)
0810 11:44:07.297533 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.297538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.298245 (+ 707us) server_negotiation.cc:407] Connection header received
0810 11:44:07.298514 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.298528 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.298648 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.298740 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.299239 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.300406 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.301848 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.302610 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.302745 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.303175 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.303316 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.303412 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.303502 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.303591 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 11:44:07.347316 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.339632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:44079 (local address 127.20.93.131:37639)
0810 11:44:07.339840 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.339848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.340638 (+ 790us) server_negotiation.cc:407] Connection header received
0810 11:44:07.340864 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.340871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.340977 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.341071 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.341860 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.342611 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.345155 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.345821 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.345938 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.346566 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.346675 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.346783 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.346907 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.346998 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:44:07.347472 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.339709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:37639 (local address 127.20.93.129:44079)
0810 11:44:07.340541 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.340557 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.340716 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.341113 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.341118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.341132 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:07.341625 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.341641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.342756 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.342761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.344674 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.344690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.346061 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.346069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.346267 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.346348 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.346794 (+ 446us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.346808 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.346887 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.347035 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":466,"threads_started":1}
I20260810 11:44:07.394590 22776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:07.394896 22776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:07.399245 22776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Sending DeleteTablet for 3 replicas of tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8
I20260810 11:44:07.400545 23231 tablet_service.cc:1514] Processing DeleteTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:07 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59390
I20260810 11:44:07.400583 22927 tablet_service.cc:1514] Processing DeleteTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:07 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:53938
I20260810 11:44:07.401886 23081 tablet_service.cc:1514] Processing DeleteTablet for tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:07 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260810 11:44:07.410264 22776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 11:44:07.411542 23383 tablet_replica.cc:330] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: stopping tablet replica
I20260810 11:44:07.412041 23383 raft_consensus.cc:2229] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:07.412426 23383 raft_consensus.cc:2258] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 11:44:07.413028 22776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:07.413268 23382 tablet_replica.cc:330] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: stopping tablet replica
I20260810 11:44:07.413710 23382 raft_consensus.cc:2229] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:07.414381 23382 raft_consensus.cc:2258] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:07.416620 23383 ts_tablet_manager.cc:1905] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:07.422111 22927 tablet_service.cc:1467] Processing CreateTablet for tablet cc543db878f54b19b1a4050c5936a9c2 (DEFAULT_TABLE table=test-table [id=05429bd79f68476e97749146de445f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:07.422468 23231 tablet_service.cc:1467] Processing CreateTablet for tablet cc543db878f54b19b1a4050c5936a9c2 (DEFAULT_TABLE table=test-table [id=05429bd79f68476e97749146de445f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:07.422660 22927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc543db878f54b19b1a4050c5936a9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:07.422541 23080 tablet_service.cc:1467] Processing CreateTablet for tablet cc543db878f54b19b1a4050c5936a9c2 (DEFAULT_TABLE table=test-table [id=05429bd79f68476e97749146de445f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:07.423009 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc543db878f54b19b1a4050c5936a9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:07.423064 23080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc543db878f54b19b1a4050c5936a9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:07.429499 23331 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Bootstrap starting.
I20260810 11:44:07.430243 23382 ts_tablet_manager.cc:1905] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:07.431327 23331 tablet_bootstrap.cc:654] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:07.432539 23329 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Bootstrap starting.
I20260810 11:44:07.434742 23329 tablet_bootstrap.cc:654] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:07.436189 23330 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Bootstrap starting.
I20260810 11:44:07.437203 23385 tablet_replica.cc:330] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: stopping tablet replica
I20260810 11:44:07.437749 23385 raft_consensus.cc:2229] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:07.438208 23330 tablet_bootstrap.cc:654] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:07.438323 23385 raft_consensus.cc:2258] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:07.440702 23385 ts_tablet_manager.cc:1905] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:07.448961 23385 ts_tablet_manager.cc:1918] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:07.449250 23385 log.cc:1196] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/5959e7a4fbab4bc8ae76e1d549c8c9d8
I20260810 11:44:07.449913 23385 ts_tablet_manager.cc:1939] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P f11eb6294528495dbe14c75ee0ab8368: Deleting consensus metadata
I20260810 11:44:07.451921 22762 catalog_manager.cc:4841] TS f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247): tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (table test-table [id=c5abf78940d2436891dc02b0078c7dd1]) successfully deleted
I20260810 11:44:07.460922 23329 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: No bootstrap required, opened a new log
I20260810 11:44:07.461398 23383 ts_tablet_manager.cc:1918] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:07.465957 23383 log.cc:1196] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/5959e7a4fbab4bc8ae76e1d549c8c9d8
I20260810 11:44:07.466130 23329 ts_tablet_manager.cc:1397] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.004s
I20260810 11:44:07.466580 23383 ts_tablet_manager.cc:1939] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P 14267d30613e4e8189c411debda5acb8: Deleting consensus metadata
I20260810 11:44:07.467113 23329 raft_consensus.cc:348] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.467473 23329 raft_consensus.cc:374] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:07.467617 23329 raft_consensus.cc:729] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Initialized, Role: FOLLOWER
I20260810 11:44:07.467976 23329 consensus_queue.cc:260] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.468389 22762 catalog_manager.cc:4841] TS 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543): tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (table test-table [id=c5abf78940d2436891dc02b0078c7dd1]) successfully deleted
I20260810 11:44:07.469521 23330 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: No bootstrap required, opened a new log
I20260810 11:44:07.469779 23330 ts_tablet_manager.cc:1397] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.008s
I20260810 11:44:07.469913 23331 tablet_bootstrap.cc:492] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: No bootstrap required, opened a new log
I20260810 11:44:07.470096 23331 ts_tablet_manager.cc:1397] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.000s
I20260810 11:44:07.470705 23330 raft_consensus.cc:348] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.470825 23331 raft_consensus.cc:348] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.471017 23330 raft_consensus.cc:374] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:07.471091 23331 raft_consensus.cc:374] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:07.471176 23331 raft_consensus.cc:729] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Initialized, Role: FOLLOWER
I20260810 11:44:07.471212 23330 raft_consensus.cc:729] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Initialized, Role: FOLLOWER
I20260810 11:44:07.471536 23330 consensus_queue.cc:260] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.471549 23331 consensus_queue.cc:260] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.472158 23330 ts_tablet_manager.cc:1428] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 11:44:07.472767 23329 ts_tablet_manager.cc:1428] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260810 11:44:07.474222 23382 ts_tablet_manager.cc:1918] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:07.474422 23331 ts_tablet_manager.cc:1428] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:07.474475 23382 log.cc:1196] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/5959e7a4fbab4bc8ae76e1d549c8c9d8
I20260810 11:44:07.475217 23382 ts_tablet_manager.cc:1939] T 5959e7a4fbab4bc8ae76e1d549c8c9d8 P ecb33c2170464b5e95db929574b924dd: Deleting consensus metadata
I20260810 11:44:07.477023 22763 catalog_manager.cc:4841] TS ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639): tablet 5959e7a4fbab4bc8ae76e1d549c8c9d8 (table test-table [id=c5abf78940d2436891dc02b0078c7dd1]) successfully deleted
I20260810 11:44:07.669981 23380 raft_consensus.cc:482] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:07.670233 23380 raft_consensus.cc:504] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.671108 23380 leader_election.cc:288] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:07.671766 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc543db878f54b19b1a4050c5936a9c2" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368" is_pre_election: true
I20260810 11:44:07.672070 23103 raft_consensus.cc:2454] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 0.
I20260810 11:44:07.672019 23254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc543db878f54b19b1a4050c5936a9c2" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd" is_pre_election: true
I20260810 11:44:07.672305 23254 raft_consensus.cc:2454] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 0.
I20260810 11:44:07.672642 22872 leader_election.cc:302] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:07.673080 23380 raft_consensus.cc:2781] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:07.673224 23380 raft_consensus.cc:482] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:07.673326 23380 raft_consensus.cc:3037] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.676967 23380 raft_consensus.cc:504] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.678225 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc543db878f54b19b1a4050c5936a9c2" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368"
I20260810 11:44:07.678318 23254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc543db878f54b19b1a4050c5936a9c2" candidate_uuid: "14267d30613e4e8189c411debda5acb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd"
I20260810 11:44:07.678479 23103 raft_consensus.cc:3037] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.678576 23254 raft_consensus.cc:3037] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:07.682360 23254 raft_consensus.cc:2454] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 1.
I20260810 11:44:07.682915 22873 leader_election.cc:302] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, ecb33c2170464b5e95db929574b924dd; no voters:
I20260810 11:44:07.683293 23355 raft_consensus.cc:2781] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:07.683348 23103 raft_consensus.cc:2454] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14267d30613e4e8189c411debda5acb8 in term 1.
I20260810 11:44:07.683461 23355 raft_consensus.cc:686] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 1 LEADER]: Becoming Leader. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Running, Role: LEADER
I20260810 11:44:07.683771 23355 consensus_queue.cc:237] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:07.684628 23380 leader_election.cc:288] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [CANDIDATE]: Term 1 election: Requested vote from peers f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:07.686952 22779 catalog_manager.cc:5495] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14267d30613e4e8189c411debda5acb8 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "14267d30613e4e8189c411debda5acb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:07.760468 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.755075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.20.93.129:36543)
0810 11:44:07.755243 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.755250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.755446 (+ 196us) server_negotiation.cc:407] Connection header received
0810 11:44:07.755664 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.755682 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.755754 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.755849 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.756236 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.756962 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.758715 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.759245 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.759401 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.759797 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.759895 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.759993 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.760121 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.760197 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 11:44:07.774722 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.769340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:39343 (local address 127.20.93.129:36543)
0810 11:44:07.769554 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:07.769559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:07.769689 (+ 130us) server_negotiation.cc:407] Connection header received
0810 11:44:07.769858 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:07.769861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:07.769936 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:07.770008 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:07.770399 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.771082 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.772927 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.773494 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.773666 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.774062 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.774178 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.774247 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:07.774421 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 11:44:07.774483 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:44:07.774943 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:07.769353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.20.93.129:39343)
0810 11:44:07.769592 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:07.769615 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:07.769747 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:07.770050 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:07.770056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:07.770069 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:07.770264 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.770275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.771218 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.771224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.772782 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:07.772797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:07.773687 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:07.773693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:07.773859 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:07.773911 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:07.774286 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:07.774301 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 11:44:07.774397 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 11:44:07.774577 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 11:44:07.783672 23254 raft_consensus.cc:1261] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Refusing update from remote peer 14267d30613e4e8189c411debda5acb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:07.783751 23103 raft_consensus.cc:1261] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Refusing update from remote peer 14267d30613e4e8189c411debda5acb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:07.784292 23380 consensus_queue.cc:1035] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.784602 23355 consensus_queue.cc:1035] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:07.792356 23361 mvcc.cc:205] Tried to move back new op lower bound from 7316939766917373952 to 7316939766515482624. Current Snapshot: MvccSnapshot[applied={T|T < 7316939766917373952}]
W20260810 11:44:07.812018 23360 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 11:44:09.417496 22779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:09.417752 22779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:09.422223 22779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Sending DeleteTablet for 3 replicas of tablet cc543db878f54b19b1a4050c5936a9c2
I20260810 11:44:09.423121 22927 tablet_service.cc:1514] Processing DeleteTablet for tablet cc543db878f54b19b1a4050c5936a9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:53938
I20260810 11:44:09.423269 23081 tablet_service.cc:1514] Processing DeleteTablet for tablet cc543db878f54b19b1a4050c5936a9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260810 11:44:09.423323 23231 tablet_service.cc:1514] Processing DeleteTablet for tablet cc543db878f54b19b1a4050c5936a9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59390
I20260810 11:44:09.435832 23414 tablet_replica.cc:330] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: stopping tablet replica
I20260810 11:44:09.436671 23416 tablet_replica.cc:330] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: stopping tablet replica
I20260810 11:44:09.437423 23414 raft_consensus.cc:2229] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:09.437719 23416 raft_consensus.cc:2229] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:09.438060 23416 raft_consensus.cc:2258] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:09.438589 23417 tablet_replica.cc:330] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: stopping tablet replica
I20260810 11:44:09.438827 23417 raft_consensus.cc:2229] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:09.438877 23414 raft_consensus.cc:2258] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:09.439098 23417 raft_consensus.cc:2258] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:09.440145 23414 ts_tablet_manager.cc:1905] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.440572 23417 ts_tablet_manager.cc:1905] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.447697 23417 ts_tablet_manager.cc:1918] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:09.447855 23417 log.cc:1196] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/cc543db878f54b19b1a4050c5936a9c2
I20260810 11:44:09.448299 23417 ts_tablet_manager.cc:1939] T cc543db878f54b19b1a4050c5936a9c2 P f11eb6294528495dbe14c75ee0ab8368: Deleting consensus metadata
I20260810 11:44:09.449409 23414 ts_tablet_manager.cc:1918] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:09.449522 22762 catalog_manager.cc:4841] TS f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247): tablet cc543db878f54b19b1a4050c5936a9c2 (table test-table [id=05429bd79f68476e97749146de445f40]) successfully deleted
I20260810 11:44:09.449738 23414 log.cc:1196] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/cc543db878f54b19b1a4050c5936a9c2
I20260810 11:44:09.439316 23416 ts_tablet_manager.cc:1905] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.450393 23414 ts_tablet_manager.cc:1939] T cc543db878f54b19b1a4050c5936a9c2 P 14267d30613e4e8189c411debda5acb8: Deleting consensus metadata
I20260810 11:44:09.451635 22762 catalog_manager.cc:4841] TS 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543): tablet cc543db878f54b19b1a4050c5936a9c2 (table test-table [id=05429bd79f68476e97749146de445f40]) successfully deleted
I20260810 11:44:09.456964 23416 ts_tablet_manager.cc:1918] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:09.457111 23416 log.cc:1196] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/cc543db878f54b19b1a4050c5936a9c2
I20260810 11:44:09.457607 23416 ts_tablet_manager.cc:1939] T cc543db878f54b19b1a4050c5936a9c2 P ecb33c2170464b5e95db929574b924dd: Deleting consensus metadata
I20260810 11:44:09.458792 22763 catalog_manager.cc:4841] TS ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639): tablet cc543db878f54b19b1a4050c5936a9c2 (table test-table [id=05429bd79f68476e97749146de445f40]) successfully deleted
Aug 10 11:44:09 dist-test-slave-njxd krb5kdc[22729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:44:09 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:09.490316 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.475382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.20.93.190:37551)
0810 11:44:09.476094 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.476099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.476123 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:09.476185 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.476189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.476261 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.476360 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:09.477783 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.478562 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.479821 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.480266 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.480365 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.484772 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:09.484778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:09.484781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:09.484822 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:09.487647 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:09.488357 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:09.488361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:09.488363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:09.488505 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:09.488859 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:09.488862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:09.488864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:09.489151 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:09.489342 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.489671 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.489923 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":328,"threads_started":1}
I20260810 11:44:09.494115 22779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:09.494837 22779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:09.501961 22927 tablet_service.cc:1467] Processing CreateTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (DEFAULT_TABLE table=test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.502134 23081 tablet_service.cc:1467] Processing CreateTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (DEFAULT_TABLE table=test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.502481 22927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.502544 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.502522 23231 tablet_service.cc:1467] Processing CreateTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (DEFAULT_TABLE table=test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.502868 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.508986 23429 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Bootstrap starting.
I20260810 11:44:09.510156 23430 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Bootstrap starting.
I20260810 11:44:09.511145 23429 tablet_bootstrap.cc:654] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.512192 23430 tablet_bootstrap.cc:654] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.514055 23432 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Bootstrap starting.
I20260810 11:44:09.515215 23429 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: No bootstrap required, opened a new log
I20260810 11:44:09.515341 23430 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: No bootstrap required, opened a new log
I20260810 11:44:09.515509 23430 ts_tablet_manager.cc:1397] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:44:09.515542 23429 ts_tablet_manager.cc:1397] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260810 11:44:09.515971 23432 tablet_bootstrap.cc:654] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.516456 23430 raft_consensus.cc:348] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.516741 23430 raft_consensus.cc:374] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.516636 23429 raft_consensus.cc:348] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.516917 23429 raft_consensus.cc:374] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.516934 23430 raft_consensus.cc:729] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.517026 23429 raft_consensus.cc:729] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.517306 23430 consensus_queue.cc:260] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.517333 23429 consensus_queue.cc:260] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.518251 23430 ts_tablet_manager.cc:1428] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 11:44:09.518658 23432 tablet_bootstrap.cc:492] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: No bootstrap required, opened a new log
I20260810 11:44:09.518810 23432 ts_tablet_manager.cc:1397] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:44:09.519734 23429 ts_tablet_manager.cc:1428] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 11:44:09.519718 23432 raft_consensus.cc:348] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.519971 23432 raft_consensus.cc:374] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.520064 23432 raft_consensus.cc:729] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.520323 23432 consensus_queue.cc:260] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.520960 23432 ts_tablet_manager.cc:1428] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 11:44:09.644779 23436 raft_consensus.cc:482] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:09.645105 23436 raft_consensus.cc:504] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.647552 23436 leader_election.cc:288] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:09.659646 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.648054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:37639 (local address 127.20.93.130:32821)
0810 11:44:09.648764 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:09.648792 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:09.648963 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:09.650793 (+ 1830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:09.650802 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:09.650843 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:09.651358 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.651376 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.654169 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.654178 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.656326 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.656344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.658502 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.658509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.658773 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.658812 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:44:09.658931 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 11:44:09.659102 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":769,"threads_started":1}
I20260810 11:44:09.659643 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.648447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:32821 (local address 127.20.93.131:37639)
0810 11:44:09.650279 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.650289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.650325 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:44:09.650431 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.650439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.650549 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.651607 (+ 1058us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:09.651882 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.653458 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.656534 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.658267 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.658501 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.658618 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.658994 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.659213 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1614,"thread_start_us":1379,"threads_started":1}
I20260810 11:44:09.660893 23253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c393dd4fc8e34d31abf3bd2b0c2f71f0" candidate_uuid: "f11eb6294528495dbe14c75ee0ab8368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd" is_pre_election: true
I20260810 11:44:09.661243 23253 raft_consensus.cc:2454] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f11eb6294528495dbe14c75ee0ab8368 in term 0.
I20260810 11:44:09.661933 23029 leader_election.cc:302] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecb33c2170464b5e95db929574b924dd, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:09.662307 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.648628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.20.93.130:50957)
0810 11:44:09.654232 (+ 5604us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:09.654258 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:09.654421 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:09.655477 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:09.655485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:09.655532 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:09.656122 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.656140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.657340 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.657347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.659448 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.659464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.661304 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.661320 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.661560 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.661629 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 11:44:09.661729 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 11:44:09.661878 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5405,"thread_start_us":5195,"threads_started":1}
I20260810 11:44:09.662515 23436 raft_consensus.cc:2781] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:09.662673 23436 raft_consensus.cc:482] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:09.662751 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.648053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:50957 (local address 127.20.93.129:36543)
0810 11:44:09.655045 (+ 6992us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.655051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.655080 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:44:09.655158 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.655163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.655270 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.655401 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:09.656361 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.657176 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.659673 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.660928 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.661167 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.661248 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.661781 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.661930 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6814,"thread_start_us":6649,"threads_started":1}
I20260810 11:44:09.662812 23436 raft_consensus.cc:3037] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.665863 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c393dd4fc8e34d31abf3bd2b0c2f71f0" candidate_uuid: "f11eb6294528495dbe14c75ee0ab8368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8" is_pre_election: true
I20260810 11:44:09.666170 22948 raft_consensus.cc:2454] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f11eb6294528495dbe14c75ee0ab8368 in term 0.
I20260810 11:44:09.667258 23436 raft_consensus.cc:504] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.668143 23436 leader_election.cc:288] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [CANDIDATE]: Term 1 election: Requested vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639)
I20260810 11:44:09.668685 23253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c393dd4fc8e34d31abf3bd2b0c2f71f0" candidate_uuid: "f11eb6294528495dbe14c75ee0ab8368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecb33c2170464b5e95db929574b924dd"
I20260810 11:44:09.668665 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c393dd4fc8e34d31abf3bd2b0c2f71f0" candidate_uuid: "f11eb6294528495dbe14c75ee0ab8368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8"
I20260810 11:44:09.668929 23253 raft_consensus.cc:3037] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.668931 22948 raft_consensus.cc:3037] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.671272 23253 raft_consensus.cc:2454] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f11eb6294528495dbe14c75ee0ab8368 in term 1.
I20260810 11:44:09.671764 23029 leader_election.cc:302] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecb33c2170464b5e95db929574b924dd, f11eb6294528495dbe14c75ee0ab8368; no voters:
I20260810 11:44:09.672128 23436 raft_consensus.cc:2781] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:09.672700 22948 raft_consensus.cc:2454] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f11eb6294528495dbe14c75ee0ab8368 in term 1.
I20260810 11:44:09.672750 23436 raft_consensus.cc:686] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 1 LEADER]: Becoming Leader. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Running, Role: LEADER
I20260810 11:44:09.673298 23436 consensus_queue.cc:237] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.677709 22777 catalog_manager.cc:5495] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 reported cstate change: term changed from 0 to 1, leader changed from <none> to f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130). New cstate: current_term: 1 leader_uuid: "f11eb6294528495dbe14c75ee0ab8368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:09.724264 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.718009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.20.93.130:46247)
0810 11:44:09.718691 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.718695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.718714 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:44:09.718808 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.718811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.718886 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.719030 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:09.719463 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.720289 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.721916 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.722349 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.722448 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.723017 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.723653 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.723776 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.723893 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.723962 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":422,"threads_started":1}
I20260810 11:44:09.740578 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.735144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:44651 (local address 127.20.93.129:36543)
0810 11:44:09.735443 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.735449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.735731 (+ 282us) server_negotiation.cc:407] Connection header received
0810 11:44:09.735900 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.735904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.736030 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.736116 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:09.736663 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.737429 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.738873 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.739311 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.739414 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.739934 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.740060 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.740143 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.740290 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.740364 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:44:09.740852 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.734926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36543 (local address 127.20.93.130:44651)
0810 11:44:09.735625 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:09.735648 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:09.735782 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:09.736143 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:09.736146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:09.736156 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:09.736529 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.736539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.737556 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.737559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.738703 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.738716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.739465 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.739469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.739659 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.739718 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.740171 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.740181 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:44:09.740248 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 11:44:09.740358 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":370,"threads_started":1}
I20260810 11:44:09.749954 22948 raft_consensus.cc:1261] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Refusing update from remote peer f11eb6294528495dbe14c75ee0ab8368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:09.750149 23253 raft_consensus.cc:1261] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Refusing update from remote peer f11eb6294528495dbe14c75ee0ab8368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:09.750772 23436 consensus_queue.cc:1035] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:09.751097 23442 consensus_queue.cc:1035] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:09.783927 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.778392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46213 (local address 127.20.93.130:46247)
0810 11:44:09.778970 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.778975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.778991 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:44:09.779045 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.779048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.779117 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.779196 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:09.779599 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.780337 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.782032 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.782632 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.782772 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.783174 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.783294 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.783399 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.783554 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.783636 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475}
I20260810 11:44:09.784075 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.778207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46247 (local address 127.20.93.129:46213)
0810 11:44:09.778795 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:09.778812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:09.778917 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:09.779231 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:09.779234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:09.779244 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:09.779477 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.779484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.780494 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.780498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.781874 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:09.781888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.782810 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.782817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:09.782957 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.782997 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.783403 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.783418 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 11:44:09.783526 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:44:09.783668 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"spinlock_wait_cycles":6016,"thread_start_us":405,"threads_started":1}
I20260810 11:44:09.824113 22777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:09.824457 22777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:09.828596 22777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Sending DeleteTablet for 3 replicas of tablet c393dd4fc8e34d31abf3bd2b0c2f71f0
I20260810 11:44:09.829923 23231 tablet_service.cc:1514] Processing DeleteTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59390
I20260810 11:44:09.830015 22927 tablet_service.cc:1514] Processing DeleteTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:53938
I20260810 11:44:09.830307 23416 tablet_replica.cc:330] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: stopping tablet replica
I20260810 11:44:09.830365 23081 tablet_service.cc:1514] Processing DeleteTablet for tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:09 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260810 11:44:09.830538 23416 raft_consensus.cc:2229] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:09.830657 23414 tablet_replica.cc:330] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: stopping tablet replica
I20260810 11:44:09.830703 23417 tablet_replica.cc:330] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: stopping tablet replica
I20260810 11:44:09.830776 23416 raft_consensus.cc:2258] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:09.830897 23414 raft_consensus.cc:2229] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:09.830897 23417 raft_consensus.cc:2229] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:09.831089 23414 raft_consensus.cc:2258] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:09.830914 22777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 11:44:09.831489 23417 raft_consensus.cc:2258] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 11:44:09.831579 22777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:09.832978 23414 ts_tablet_manager.cc:1905] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.833019 23416 ts_tablet_manager.cc:1905] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.840242 23081 tablet_service.cc:1467] Processing CreateTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 (DEFAULT_TABLE table=test-table [id=52eec12b4cd74ceb89440caf07c456b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.840235 23231 tablet_service.cc:1467] Processing CreateTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 (DEFAULT_TABLE table=test-table [id=52eec12b4cd74ceb89440caf07c456b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.840708 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eceb46f2ba4a779ccd9e1705c50897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.840724 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eceb46f2ba4a779ccd9e1705c50897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.845165 22927 tablet_service.cc:1467] Processing CreateTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 (DEFAULT_TABLE table=test-table [id=52eec12b4cd74ceb89440caf07c456b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:09.845659 22927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eceb46f2ba4a779ccd9e1705c50897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:09.846407 23417 ts_tablet_manager.cc:1905] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:09.854015 23417 ts_tablet_manager.cc:1918] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:09.854228 23417 log.cc:1196] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/c393dd4fc8e34d31abf3bd2b0c2f71f0
I20260810 11:44:09.854722 23417 ts_tablet_manager.cc:1939] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P f11eb6294528495dbe14c75ee0ab8368: Deleting consensus metadata
I20260810 11:44:09.855094 23432 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Bootstrap starting.
I20260810 11:44:09.855996 22762 catalog_manager.cc:4841] TS f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247): tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (table test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]) successfully deleted
I20260810 11:44:09.856992 23432 tablet_bootstrap.cc:654] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.861550 23414 ts_tablet_manager.cc:1918] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:09.861816 23430 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Bootstrap starting.
I20260810 11:44:09.863154 23429 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Bootstrap starting.
I20260810 11:44:09.863718 23430 tablet_bootstrap.cc:654] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.864123 23416 ts_tablet_manager.cc:1918] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:09.864312 23416 log.cc:1196] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/c393dd4fc8e34d31abf3bd2b0c2f71f0
I20260810 11:44:09.865024 23416 ts_tablet_manager.cc:1939] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P ecb33c2170464b5e95db929574b924dd: Deleting consensus metadata
I20260810 11:44:09.865100 23429 tablet_bootstrap.cc:654] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:09.870033 22763 catalog_manager.cc:4841] TS ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639): tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (table test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]) successfully deleted
I20260810 11:44:09.873154 23414 log.cc:1196] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/c393dd4fc8e34d31abf3bd2b0c2f71f0
I20260810 11:44:09.873812 23414 ts_tablet_manager.cc:1939] T c393dd4fc8e34d31abf3bd2b0c2f71f0 P 14267d30613e4e8189c411debda5acb8: Deleting consensus metadata
I20260810 11:44:09.874989 22762 catalog_manager.cc:4841] TS 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543): tablet c393dd4fc8e34d31abf3bd2b0c2f71f0 (table test-table [id=1b60fbcf86274c0e9ea4e12ea541392e]) successfully deleted
I20260810 11:44:09.881786 23432 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: No bootstrap required, opened a new log
I20260810 11:44:09.881882 23430 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: No bootstrap required, opened a new log
I20260810 11:44:09.882031 23432 ts_tablet_manager.cc:1397] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.009s
I20260810 11:44:09.882083 23430 ts_tablet_manager.cc:1397] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.008s
I20260810 11:44:09.883056 23432 raft_consensus.cc:348] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.883381 23432 raft_consensus.cc:374] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.883481 23432 raft_consensus.cc:729] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14267d30613e4e8189c411debda5acb8, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.883797 23432 consensus_queue.cc:260] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.884577 23430 raft_consensus.cc:348] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.884847 23430 raft_consensus.cc:374] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.884930 23430 raft_consensus.cc:729] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11eb6294528495dbe14c75ee0ab8368, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.885142 23429 tablet_bootstrap.cc:492] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: No bootstrap required, opened a new log
I20260810 11:44:09.885207 23430 consensus_queue.cc:260] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.885372 23429 ts_tablet_manager.cc:1397] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.002s
I20260810 11:44:09.885851 23430 ts_tablet_manager.cc:1428] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 11:44:09.886296 23429 raft_consensus.cc:348] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.886607 23429 raft_consensus.cc:374] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:09.886739 23429 raft_consensus.cc:729] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Initialized, Role: FOLLOWER
I20260810 11:44:09.887085 23429 consensus_queue.cc:260] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.889092 23429 ts_tablet_manager.cc:1428] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 11:44:09.893237 23432 ts_tablet_manager.cc:1428] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260810 11:44:09.941327 23437 raft_consensus.cc:482] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:09.941642 23437 raft_consensus.cc:504] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.942515 23437 leader_election.cc:288] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247)
I20260810 11:44:09.943257 23102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eceb46f2ba4a779ccd9e1705c50897" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368" is_pre_election: true
I20260810 11:44:09.943266 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eceb46f2ba4a779ccd9e1705c50897" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8" is_pre_election: true
I20260810 11:44:09.943555 23102 raft_consensus.cc:2454] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 0.
I20260810 11:44:09.943580 22948 raft_consensus.cc:2454] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 0.
I20260810 11:44:09.944156 23180 leader_election.cc:302] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, ecb33c2170464b5e95db929574b924dd; no voters:
I20260810 11:44:09.944849 23437 raft_consensus.cc:2781] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:09.944983 23437 raft_consensus.cc:482] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:09.945079 23437 raft_consensus.cc:3037] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.947516 23437 raft_consensus.cc:504] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.948179 23437 leader_election.cc:288] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 election: Requested vote from peers 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543), f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247)
I20260810 11:44:09.948699 22948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eceb46f2ba4a779ccd9e1705c50897" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14267d30613e4e8189c411debda5acb8"
I20260810 11:44:09.948980 22948 raft_consensus.cc:3037] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.948869 23102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eceb46f2ba4a779ccd9e1705c50897" candidate_uuid: "ecb33c2170464b5e95db929574b924dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11eb6294528495dbe14c75ee0ab8368"
I20260810 11:44:09.949164 23102 raft_consensus.cc:3037] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:09.951572 22948 raft_consensus.cc:2454] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 1.
I20260810 11:44:09.952209 23180 leader_election.cc:302] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14267d30613e4e8189c411debda5acb8, ecb33c2170464b5e95db929574b924dd; no voters:
I20260810 11:44:09.952543 23437 raft_consensus.cc:2781] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:09.952560 23102 raft_consensus.cc:2454] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecb33c2170464b5e95db929574b924dd in term 1.
I20260810 11:44:09.953292 23437 raft_consensus.cc:686] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 1 LEADER]: Becoming Leader. State: Replica: ecb33c2170464b5e95db929574b924dd, State: Running, Role: LEADER
I20260810 11:44:09.953693 23437 consensus_queue.cc:237] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } }
I20260810 11:44:09.956737 22779 catalog_manager.cc:5495] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd reported cstate change: term changed from 0 to 1, leader changed from <none> to ecb33c2170464b5e95db929574b924dd (127.20.93.131). New cstate: current_term: 1 leader_uuid: "ecb33c2170464b5e95db929574b924dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecb33c2170464b5e95db929574b924dd" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 37639 } health_report { overall_health: HEALTHY } } }
I20260810 11:44:09.986918 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:09.980250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.20.93.131:37639)
0810 11:44:09.980425 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:09.980430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:09.980682 (+ 252us) server_negotiation.cc:407] Connection header received
0810 11:44:09.980856 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:09.980862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:09.980958 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:09.981064 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:09.981505 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.982667 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.984675 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:09.985325 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:09.985456 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:09.986066 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:09.986239 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:09.986339 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:09.986523 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 11:44:09.986613 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 11:44:10.000324 23102 raft_consensus.cc:1261] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Refusing update from remote peer ecb33c2170464b5e95db929574b924dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:10.000528 22948 raft_consensus.cc:1261] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Refusing update from remote peer ecb33c2170464b5e95db929574b924dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:10.001010 23459 consensus_queue.cc:1035] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11eb6294528495dbe14c75ee0ab8368" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:10.001413 23437 consensus_queue.cc:1035] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "14267d30613e4e8189c411debda5acb8" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:44:10.023100 23397 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 11:44:11.620505 22779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:11.620754 22779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:11.624598 22779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299949918c2e421fa28a13a362014253: Sending DeleteTablet for 3 replicas of tablet 47eceb46f2ba4a779ccd9e1705c50897
I20260810 11:44:11.625399 22927 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:53938
I20260810 11:44:11.625613 23231 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59390
I20260810 11:44:11.625672 23081 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eceb46f2ba4a779ccd9e1705c50897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260810 11:44:11.626312 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22855
I20260810 11:44:11.636443 23491 tablet_replica.cc:330] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: stopping tablet replica
I20260810 11:44:11.636770 23491 raft_consensus.cc:2229] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:11.637272 23491 raft_consensus.cc:2258] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:11.638592 23491 ts_tablet_manager.cc:1905] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:11.641417 23490 tablet_replica.cc:330] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: stopping tablet replica
I20260810 11:44:11.641544 23492 tablet_replica.cc:330] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: stopping tablet replica
I20260810 11:44:11.641688 23490 raft_consensus.cc:2229] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:11.641786 23492 raft_consensus.cc:2229] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:11.641980 23490 raft_consensus.cc:2258] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:11.642057 23492 raft_consensus.cc:2258] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:11.643141 23490 ts_tablet_manager.cc:1905] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:11.643149 23492 ts_tablet_manager.cc:1905] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:11.647014 23491 ts_tablet_manager.cc:1918] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:11.647244 23491 log.cc:1196] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/47eceb46f2ba4a779ccd9e1705c50897
I20260810 11:44:11.647791 23491 ts_tablet_manager.cc:1939] T 47eceb46f2ba4a779ccd9e1705c50897 P ecb33c2170464b5e95db929574b924dd: Deleting consensus metadata
I20260810 11:44:11.648994 22763 catalog_manager.cc:4841] TS ecb33c2170464b5e95db929574b924dd (127.20.93.131:37639): tablet 47eceb46f2ba4a779ccd9e1705c50897 (table test-table [id=52eec12b4cd74ceb89440caf07c456b0]) successfully deleted
I20260810 11:44:11.649725 23490 ts_tablet_manager.cc:1918] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:11.649891 23490 log.cc:1196] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/47eceb46f2ba4a779ccd9e1705c50897
I20260810 11:44:11.649873 23492 ts_tablet_manager.cc:1918] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:44:11.650017 23492 log.cc:1196] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/47eceb46f2ba4a779ccd9e1705c50897
I20260810 11:44:11.650378 23490 ts_tablet_manager.cc:1939] T 47eceb46f2ba4a779ccd9e1705c50897 P 14267d30613e4e8189c411debda5acb8: Deleting consensus metadata
I20260810 11:44:11.650460 23492 ts_tablet_manager.cc:1939] T 47eceb46f2ba4a779ccd9e1705c50897 P f11eb6294528495dbe14c75ee0ab8368: Deleting consensus metadata
I20260810 11:44:11.651419 22762 catalog_manager.cc:4841] TS 14267d30613e4e8189c411debda5acb8 (127.20.93.129:36543): tablet 47eceb46f2ba4a779ccd9e1705c50897 (table test-table [id=52eec12b4cd74ceb89440caf07c456b0]) successfully deleted
I20260810 11:44:11.651688 22762 catalog_manager.cc:4841] TS f11eb6294528495dbe14c75ee0ab8368 (127.20.93.130:46247): tablet 47eceb46f2ba4a779ccd9e1705c50897 (table test-table [id=52eec12b4cd74ceb89440caf07c456b0]) successfully deleted
Aug 10 11:44:11 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:11.662468 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:11.629209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.20.93.129:36543)
0810 11:44:11.639769 (+ 10560us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:11.639775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:11.639799 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:11.639864 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:11.639869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:11.639957 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:11.640187 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:11.642257 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.643369 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.645311 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.646027 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.646146 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:11.656191 (+ 10045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:11.656215 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:11.656232 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:11.656280 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:11.659871 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.660592 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.660600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.660602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.660749 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.661091 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.661094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.661096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.661453 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:11.661765 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:11.662004 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 11:44:11.662180 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10429,"thread_start_us":10338,"threads_started":1}
W20260810 11:44:11.663157 23000 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46824
W20260810 11:44:11.663870 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22855: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:11.664121 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22855
I20260810 11:44:11.695667 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23010
Aug 10 11:44:11 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:11.711406 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:11.696489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.20.93.130:46247)
0810 11:44:11.697085 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:11.697090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:11.697130 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:44:11.697204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:11.697208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:11.697284 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:11.697374 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:11.698819 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.699615 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.700808 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.701202 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.701313 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:11.705721 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:11.705741 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:11.705766 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:11.705820 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:11.708585 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.709287 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.709294 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.709296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.709422 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.709946 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.709949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.709951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.710329 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:11.710560 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:11.710831 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 11:44:11.711002 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":317,"threads_started":1}
W20260810 11:44:11.712224 23153 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49686
W20260810 11:44:11.712904 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23010: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:11.713078 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23010
I20260810 11:44:11.739116 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23162
Aug 10 11:44:11 dist-test-slave-njxd krb5kdc[22729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:11.754606 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:11.739840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.20.93.131:37639)
0810 11:44:11.740360 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:11.740364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:11.740380 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:44:11.740432 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:11.740436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:11.740501 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:11.740606 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:11.742028 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.742764 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.743886 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:11.744289 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:11.744382 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:11.748921 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:11.748936 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:11.748952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:11.748988 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:11.751839 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.752581 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.752588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.752590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.752722 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:11.753100 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:11.753104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:11.753107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:11.753465 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:11.753774 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:11.754019 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 11:44:11.754196 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":277,"threads_started":1}
W20260810 11:44:11.755326 23302 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50852
W20260810 11:44:11.755947 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23162: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:11.756107 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23162
I20260810 11:44:11.781837 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 22743
I20260810 11:44:11.782994 22820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:11.903669 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 22743
2026-08-10T11:44:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8949 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:12 dist-test-slave-njxd krb5kdc[23501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:12 dist-test-slave-njxd krb5kdc[23501](info): set up 2 sockets
Aug 10 11:44:12 dist-test-slave-njxd krb5kdc[23501](info): commencing operation
krb5kdc: starting...
W20260810 11:44:13.052996 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:13 dist-test-slave-njxd krb5kdc[23501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:13.290092 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33407
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:33407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:13.630379 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:13.630772 23515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:13.630930 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:13.638331 23515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:13.638458 23515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:13.638506 23515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:13.638566 23515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:13.638612 23515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:13.648902 23515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:33407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:13.650301 23515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:13.651789 23515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:13.664690 23522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:13.664784 23524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:13.664737 23521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:13.665647 23515 server_base.cc:1034] running on GCE node
I20260810 11:44:13.666538 23515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:13.668040 23515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:13.669256 23515 hybrid_clock.cc:648] HybridClock initialized: now 1786362253669209 us; error 36 us; skew 500 ppm
I20260810 11:44:13.669708 23515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:13 dist-test-slave-njxd krb5kdc[23501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362253, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:13.673043 23515 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:13.674700 23515 webserver.cc:460] Webserver started at http://127.20.93.190:45817/ using document root <none> and password file <none>
I20260810 11:44:13.675457 23515 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:13.675592 23515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:13.675941 23515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:13.679138 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "c5692684101f41ddbb5d444eb2fae55b"
format_stamp: "Formatted at 2026-08-10 11:44:13 on dist-test-slave-njxd"
server_key: "a5ecc8197ec604f931a906c08bc3fd34"
server_key_iv: "689ffbf174d9115174f8eff46f2ae89d"
server_key_version: "encryptionkey@0"
I20260810 11:44:13.679970 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "c5692684101f41ddbb5d444eb2fae55b"
format_stamp: "Formatted at 2026-08-10 11:44:13 on dist-test-slave-njxd"
server_key: "a5ecc8197ec604f931a906c08bc3fd34"
server_key_iv: "689ffbf174d9115174f8eff46f2ae89d"
server_key_version: "encryptionkey@0"
I20260810 11:44:13.686375 23515 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.007s
I20260810 11:44:13.691721 23531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:13.693571 23515 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:13.693827 23515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "c5692684101f41ddbb5d444eb2fae55b"
format_stamp: "Formatted at 2026-08-10 11:44:13 on dist-test-slave-njxd"
server_key: "a5ecc8197ec604f931a906c08bc3fd34"
server_key_iv: "689ffbf174d9115174f8eff46f2ae89d"
server_key_version: "encryptionkey@0"
I20260810 11:44:13.694093 23515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:13.715876 23515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:13.720309 23515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:13.720630 23515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:13.740288 23515 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:33407
I20260810 11:44:13.740307 23587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:33407 every 8 connection(s)
I20260810 11:44:13.741827 23515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:13.743377 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23515
I20260810 11:44:13.743623 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:13.744536 20854 external_mini_cluster.cc:1428] Setting key 8fc6e23354ec2ed31b832ceaa1e9d71e
I20260810 11:44:13.747052 23588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:44:13 dist-test-slave-njxd krb5kdc[23501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:13.760758 23588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b: Bootstrap starting.
I20260810 11:44:13.764320 23588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:13.765549 23588 log.cc:824] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:13.768245 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:13.746751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.20.93.190:33407)
0810 11:44:13.747573 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:13.747585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:13.747627 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:44:13.749287 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:13.749310 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:13.749801 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:13.750155 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:13.752053 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:13.753406 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:13.755197 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:13.755859 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:13.755984 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:13.760754 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:13.760762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:13.760776 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:13.760817 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:13.763924 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:13.764724 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:13.764731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:13.764742 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:13.764909 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:13.765395 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:13.765399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:13.765401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:13.766060 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:13.766449 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:13.766878 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 11:44:13.767282 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":399,"threads_started":1}
I20260810 11:44:13.768780 23588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b: No bootstrap required, opened a new log
I20260810 11:44:13.776865 23588 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } }
I20260810 11:44:13.777284 23588 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:13.777398 23588 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5692684101f41ddbb5d444eb2fae55b, State: Initialized, Role: FOLLOWER
I20260810 11:44:13.778091 23588 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } }
I20260810 11:44:13.778347 23588 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:13.778503 23588 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:13.778695 23588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:13.780936 23588 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } }
I20260810 11:44:13.781455 23588 leader_election.cc:302] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5692684101f41ddbb5d444eb2fae55b; no voters:
I20260810 11:44:13.782224 23588 leader_election.cc:288] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:13.782383 23593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:13.783397 23593 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [term 1 LEADER]: Becoming Leader. State: Replica: c5692684101f41ddbb5d444eb2fae55b, State: Running, Role: LEADER
I20260810 11:44:13.784022 23593 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } }
I20260810 11:44:13.784907 23588 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:13.787263 23594 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5692684101f41ddbb5d444eb2fae55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } } }
I20260810 11:44:13.787142 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5692684101f41ddbb5d444eb2fae55b. Latest consensus state: current_term: 1 leader_uuid: "c5692684101f41ddbb5d444eb2fae55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5692684101f41ddbb5d444eb2fae55b" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33407 } } }
I20260810 11:44:13.787798 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:13.787858 23594 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:13.789705 23601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:13.794782 23601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:13.804085 23601 catalog_manager.cc:1347] Generated new cluster ID: 4989e68630914da6b5265abc9ee27a6e
I20260810 11:44:13.804234 23601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:13.814927 23601 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:13.816979 23601 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:13.826095 23601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5692684101f41ddbb5d444eb2fae55b: Generated new TSK 0
I20260810 11:44:13.827335 23601 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:13.885565 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:14.234256 23616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:14.234623 23616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:14.234800 23616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:14.242026 23616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:14.242184 23616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:14.242372 23616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:14.252136 23616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:14.253523 23616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:14.254966 23616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:14.270262 23621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:14.270347 23622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:14.270298 23624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:14.271319 23616 server_base.cc:1034] running on GCE node
I20260810 11:44:14.272010 23616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:14.273061 23616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:14.274300 23616 hybrid_clock.cc:648] HybridClock initialized: now 1786362254274243 us; error 51 us; skew 500 ppm
I20260810 11:44:14.274675 23616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:14 dist-test-slave-njxd krb5kdc[23501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362254, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:14.278069 23616 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:14.279683 23616 webserver.cc:460] Webserver started at http://127.20.93.129:44363/ using document root <none> and password file <none>
I20260810 11:44:14.280453 23616 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:14.280611 23616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:14.280961 23616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:14.284194 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "47bc7fb425cd4a24965a1acb0cead78b"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "2ccda2228601d24620dafa62da24c700"
server_key_iv: "7487d8585afbd173a6d155af601f856e"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.285167 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "47bc7fb425cd4a24965a1acb0cead78b"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "2ccda2228601d24620dafa62da24c700"
server_key_iv: "7487d8585afbd173a6d155af601f856e"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.291471 23616 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:44:14.295431 23633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.297073 23616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:14.297276 23616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "47bc7fb425cd4a24965a1acb0cead78b"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "2ccda2228601d24620dafa62da24c700"
server_key_iv: "7487d8585afbd173a6d155af601f856e"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.297519 23616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:14.327817 23616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:14.332271 23616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:14.332607 23616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:14.333670 23616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:14.335127 23616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:14.335244 23616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.335400 23616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:14.335484 23616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.370931 23616 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:42569
I20260810 11:44:14.370981 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:42569 every 8 connection(s)
I20260810 11:44:14.372344 23616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:14.382102 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23616
I20260810 11:44:14.382330 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:14.382982 20854 external_mini_cluster.cc:1428] Setting key 06e78808ac2bf86c0af0d048f00eed2a
Aug 10 11:44:14 dist-test-slave-njxd krb5kdc[23501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362254, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:14.395741 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:14.375469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:36691 (local address 127.20.93.190:33407)
0810 11:44:14.375955 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:14.375962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:14.377052 (+ 1090us) server_negotiation.cc:407] Connection header received
0810 11:44:14.378996 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:14.379001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:14.379094 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:14.379205 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:14.380775 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.381510 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.383352 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.383864 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.383973 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:14.388363 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:14.388373 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:14.388381 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:14.388432 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:14.392296 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:14.393103 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:14.393111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:14.393118 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:14.393293 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:14.393845 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:14.393853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:14.393858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:14.394284 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:14.394457 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:14.395273 (+ 816us) server_negotiation.cc:299] Negotiation successful
0810 11:44:14.395466 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":230,"threads_started":1}
I20260810 11:44:14.397445 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:14.376141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33407 (local address 127.20.93.129:36691)
0810 11:44:14.376881 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:14.376921 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:14.378708 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:14.379431 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:14.379441 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:14.380005 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:14.380609 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:14.380628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.381717 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.381721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:14.383191 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:14.383204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.384086 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.384093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:14.384371 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:14.385219 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:14.385280 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:14.388085 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:14.392473 (+ 4388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:14.392482 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:14.392496 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:14.392968 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:14.393411 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:14.393414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:14.393416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:14.393702 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:14.394425 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:14.394431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:14.394817 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 11:44:14.395102 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 11:44:14.395761 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":296,"threads_started":1}
I20260810 11:44:14.399247 23752 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33407
I20260810 11:44:14.399574 23752 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:14.400341 23752 heartbeater.cc:511] Master 127.20.93.190:33407 requested a full tablet report, sending...
I20260810 11:44:14.402798 23549 ts_manager.cc:194] Registered new tserver with Master: 47bc7fb425cd4a24965a1acb0cead78b (127.20.93.129:42569)
I20260810 11:44:14.404424 23549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:36691
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:14.437920 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:14.778116 23758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:14.778460 23758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:14.778640 23758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:14.786031 23758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:14.786190 23758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:14.786372 23758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:14.796398 23758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:14.797822 23758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:14.799302 23758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:14.812668 23766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:14.812589 23764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:14.813122 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:14.814388 23758 server_base.cc:1034] running on GCE node
I20260810 11:44:14.814986 23758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:14.815904 23758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:14.817148 23758 hybrid_clock.cc:648] HybridClock initialized: now 1786362254817104 us; error 28 us; skew 500 ppm
I20260810 11:44:14.817437 23758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:14 dist-test-slave-njxd krb5kdc[23501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362254, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:14.820699 23758 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:14.822263 23758 webserver.cc:460] Webserver started at http://127.20.93.130:38659/ using document root <none> and password file <none>
I20260810 11:44:14.822983 23758 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:14.823122 23758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:14.823457 23758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:14.826547 23758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "8fd37d1724c7423ca1c637b84e3eef08"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "0dfb49d3d479df2cdcedbe22f99b279f"
server_key_iv: "fcb853b8657e0f74d30cf9313bad2797"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.827368 23758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "8fd37d1724c7423ca1c637b84e3eef08"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "0dfb49d3d479df2cdcedbe22f99b279f"
server_key_iv: "fcb853b8657e0f74d30cf9313bad2797"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.832607 23758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:44:14.836323 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.837808 23758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:14.837996 23758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "8fd37d1724c7423ca1c637b84e3eef08"
format_stamp: "Formatted at 2026-08-10 11:44:14 on dist-test-slave-njxd"
server_key: "0dfb49d3d479df2cdcedbe22f99b279f"
server_key_iv: "fcb853b8657e0f74d30cf9313bad2797"
server_key_version: "encryptionkey@0"
I20260810 11:44:14.838235 23758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:14.850728 23758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:14.854939 23758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:14.855275 23758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:14.856319 23758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:14.857810 23758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:14.857888 23758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.858053 23758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:14.858108 23758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:14.893123 23758 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:43077
I20260810 11:44:14.893167 23891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:43077 every 8 connection(s)
I20260810 11:44:14.894636 23758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:14.901546 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23758
I20260810 11:44:14.901831 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:14.902464 20854 external_mini_cluster.cc:1428] Setting key 27d163f9fe53f506f6c79408d3b10db5
Aug 10 11:44:14 dist-test-slave-njxd krb5kdc[23501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362254, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:14.920897 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:14.897783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:45767 (local address 127.20.93.190:33407)
0810 11:44:14.898702 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:14.898709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:14.899583 (+ 874us) server_negotiation.cc:407] Connection header received
0810 11:44:14.901426 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:14.901434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:14.901538 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:14.901713 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:14.903927 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.904802 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.907433 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.907877 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.907975 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:14.913040 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:14.913051 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:14.913059 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:14.913112 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:14.917061 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:14.918089 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:14.918098 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:14.918104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:14.918285 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:14.918758 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:14.918767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:14.918773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:14.919198 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:14.919365 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:14.920116 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 11:44:14.920396 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":296,"threads_started":1}
I20260810 11:44:14.921631 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:14.898481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33407 (local address 127.20.93.130:45767)
0810 11:44:14.899411 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:14.899455 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:14.901127 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:14.902207 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:14.902220 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:14.903012 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:14.903746 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:14.903772 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.904978 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.904984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:14.907273 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:14.907298 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:14.908123 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:14.908132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:14.908486 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:14.909295 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:14.909333 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:14.912662 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:14.917276 (+ 4614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:14.917285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:14.917301 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:14.917918 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:14.918408 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:14.918412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:14.918414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:14.918623 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:14.919328 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:14.919334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:14.919687 (+ 353us) client_negotiation.cc:769] Sending connection context
0810 11:44:14.919938 (+ 251us) client_negotiation.cc:240] Negotiation successful
0810 11:44:14.920372 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":346,"threads_started":1}
I20260810 11:44:14.923297 23892 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33407
I20260810 11:44:14.923663 23892 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:14.924540 23892 heartbeater.cc:511] Master 127.20.93.190:33407 requested a full tablet report, sending...
I20260810 11:44:14.926205 23549 ts_manager.cc:194] Registered new tserver with Master: 8fd37d1724c7423ca1c637b84e3eef08 (127.20.93.130:43077)
I20260810 11:44:14.927111 23549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:45767
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:14.957175 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:15.304658 23899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:15.305016 23899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:15.305205 23899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:15.312539 23899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:15.312707 23899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:15.312896 23899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:15.323014 23899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:15.324476 23899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:15.326361 23899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:15.339639 23904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:15.339738 23908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:15.339685 23905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:15.340873 23899 server_base.cc:1034] running on GCE node
I20260810 11:44:15.341662 23899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:15.342823 23899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:15.344053 23899 hybrid_clock.cc:648] HybridClock initialized: now 1786362255344006 us; error 33 us; skew 500 ppm
I20260810 11:44:15.344420 23899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:15 dist-test-slave-njxd krb5kdc[23501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362255, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:15.347759 23899 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:15.349313 23899 webserver.cc:460] Webserver started at http://127.20.93.131:37627/ using document root <none> and password file <none>
I20260810 11:44:15.350098 23899 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:15.350216 23899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:15.350498 23899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:15.353508 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "731268853f5f4bbabed1115ea26a8d6c"
format_stamp: "Formatted at 2026-08-10 11:44:15 on dist-test-slave-njxd"
server_key: "269c5383a0656314f1ef02618691311d"
server_key_iv: "bc9c249ebceb4133fffee09374c5e833"
server_key_version: "encryptionkey@0"
I20260810 11:44:15.354373 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "731268853f5f4bbabed1115ea26a8d6c"
format_stamp: "Formatted at 2026-08-10 11:44:15 on dist-test-slave-njxd"
server_key: "269c5383a0656314f1ef02618691311d"
server_key_iv: "bc9c249ebceb4133fffee09374c5e833"
server_key_version: "encryptionkey@0"
I20260810 11:44:15.359818 23899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:15.363389 23915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:15.364781 23899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:15.364920 23899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "731268853f5f4bbabed1115ea26a8d6c"
format_stamp: "Formatted at 2026-08-10 11:44:15 on dist-test-slave-njxd"
server_key: "269c5383a0656314f1ef02618691311d"
server_key_iv: "bc9c249ebceb4133fffee09374c5e833"
server_key_version: "encryptionkey@0"
I20260810 11:44:15.365094 23899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:15.377249 23899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:15.381176 23899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:15.381443 23899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:15.382443 23899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:15.383754 23899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:15.383826 23899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:15.383909 23899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:15.383952 23899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:15.407629 23752 heartbeater.cc:503] Master 127.20.93.190:33407 was elected leader, sending a full tablet report...
I20260810 11:44:15.419329 23899 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41621
I20260810 11:44:15.419368 24033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41621 every 8 connection(s)
I20260810 11:44:15.420728 23899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:15.426860 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 23899
I20260810 11:44:15.427074 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:15.427620 20854 external_mini_cluster.cc:1428] Setting key 0cb679a98a4f493edbc5284bacbb1b37
Aug 10 11:44:15 dist-test-slave-njxd krb5kdc[23501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362255, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:15.441557 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:15.423737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:47363 (local address 127.20.93.190:33407)
0810 11:44:15.424224 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:15.424230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:15.425111 (+ 881us) server_negotiation.cc:407] Connection header received
0810 11:44:15.427079 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:15.427088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:15.427220 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:15.427348 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:15.429235 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:15.430210 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:15.431981 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:15.432397 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:15.432496 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:15.435936 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:15.435941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:15.435944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:15.435979 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:15.438626 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:15.439389 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:15.439393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:15.439395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:15.439518 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:15.439902 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:15.439905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:15.439907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:15.440169 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:15.440296 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:15.440950 (+ 654us) server_negotiation.cc:299] Negotiation successful
0810 11:44:15.441168 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":271,"threads_started":1}
I20260810 11:44:15.443017 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:15.424231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33407 (local address 127.20.93.131:47363)
0810 11:44:15.424948 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:15.424987 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:15.426763 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:15.427547 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:15.427556 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:15.428163 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:15.429032 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:15.429062 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:15.430362 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:15.430369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:15.431840 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:15.431858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:15.432590 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:15.432594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:15.432800 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:15.433437 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:15.433458 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:15.435720 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:15.438804 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:15.438811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:15.438826 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:15.439285 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:15.439617 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:15.439620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:15.439622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:15.439809 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:15.440289 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:15.440296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:15.440619 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 11:44:15.440863 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 11:44:15.441301 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":282,"threads_started":1}
I20260810 11:44:15.444483 24034 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33407
I20260810 11:44:15.444844 24034 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:15.445631 24034 heartbeater.cc:511] Master 127.20.93.190:33407 requested a full tablet report, sending...
I20260810 11:44:15.447113 23549 ts_manager.cc:194] Registered new tserver with Master: 731268853f5f4bbabed1115ea26a8d6c (127.20.93.131:41621)
I20260810 11:44:15.447902 23549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:47363
I20260810 11:44:15.456044 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:44:15.495066 24036 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:15.492333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.20.93.190:33407)
0810 11:44:15.492522 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:15.492527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:15.493070 (+ 543us) server_negotiation.cc:407] Connection header received
0810 11:44:15.493232 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:15.493238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:15.493306 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:15.493407 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:15.494763 (+ 1356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54356: BlockingRecv error: recv got EOF from 127.0.0.1:54356 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:44:15.498335 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23616
W20260810 11:44:15.501277 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23616: Not authorized: Client connection negotiation failed: client connection to 127.20.93.129:42569: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:44:15.501487 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23616
I20260810 11:44:15.520102 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23758
W20260810 11:44:15.523375 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23758: Not authorized: Client connection negotiation failed: client connection to 127.20.93.130:43077: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:44:15.523577 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23758
I20260810 11:44:15.543221 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23899
W20260810 11:44:15.546494 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23899: Not authorized: Client connection negotiation failed: client connection to 127.20.93.131:41621: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:44:15.546674 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23899
I20260810 11:44:15.566058 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 23515
I20260810 11:44:15.567387 23582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:15.673822 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 23515
2026-08-10T11:44:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3768 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:15 dist-test-slave-njxd krb5kdc[24054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:15 dist-test-slave-njxd krb5kdc[24054](info): set up 2 sockets
Aug 10 11:44:15 dist-test-slave-njxd krb5kdc[24054](info): commencing operation
krb5kdc: starting...
W20260810 11:44:16.850311 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:16 dist-test-slave-njxd krb5kdc[24054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:17.087505 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:32863
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:32863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:17.421461 24067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:17.421800 24067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:17.421891 24067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:17.429257 24067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:17.429381 24067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:17.429428 24067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:17.429484 24067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:17.429531 24067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:17.439992 24067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:32863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:17.441354 24067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:17.442709 24067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:17.455528 24076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:17.455482 24073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:17.455571 24074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:17.456419 24067 server_base.cc:1034] running on GCE node
I20260810 11:44:17.457221 24067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:17.458775 24067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:17.460021 24067 hybrid_clock.cc:648] HybridClock initialized: now 1786362257459970 us; error 48 us; skew 500 ppm
I20260810 11:44:17.460409 24067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:17 dist-test-slave-njxd krb5kdc[24054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362257, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:17.466884 24067 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:17.468528 24067 webserver.cc:460] Webserver started at http://127.20.93.190:46019/ using document root <none> and password file <none>
I20260810 11:44:17.469209 24067 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:17.469311 24067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:17.469731 24067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:17.472741 24067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "fc45120139164e6d98434464ff77075f"
format_stamp: "Formatted at 2026-08-10 11:44:17 on dist-test-slave-njxd"
server_key: "3d271bc4323f7f00e577a897cfaa2eb3"
server_key_iv: "7a4cd000eccfb33d88ecdadab8b92732"
server_key_version: "encryptionkey@0"
I20260810 11:44:17.473480 24067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "fc45120139164e6d98434464ff77075f"
format_stamp: "Formatted at 2026-08-10 11:44:17 on dist-test-slave-njxd"
server_key: "3d271bc4323f7f00e577a897cfaa2eb3"
server_key_iv: "7a4cd000eccfb33d88ecdadab8b92732"
server_key_version: "encryptionkey@0"
I20260810 11:44:17.478905 24067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:44:17.482663 24083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:17.484107 24067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:44:17.484258 24067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "fc45120139164e6d98434464ff77075f"
format_stamp: "Formatted at 2026-08-10 11:44:17 on dist-test-slave-njxd"
server_key: "3d271bc4323f7f00e577a897cfaa2eb3"
server_key_iv: "7a4cd000eccfb33d88ecdadab8b92732"
server_key_version: "encryptionkey@0"
I20260810 11:44:17.484431 24067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:17.504953 24067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:17.509212 24067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:17.509466 24067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:17.529393 24067 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:32863
I20260810 11:44:17.529428 24135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:32863 every 8 connection(s)
I20260810 11:44:17.530941 24067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:17.535079 24136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:17.541152 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24067
I20260810 11:44:17.541399 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:17.542111 20854 external_mini_cluster.cc:1428] Setting key 170d31ee1815552acf5d82bde5800499
I20260810 11:44:17.547935 24136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f: Bootstrap starting.
I20260810 11:44:17.552062 24136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:17.553344 24136 log.cc:824] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:17.556802 24136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f: No bootstrap required, opened a new log
Aug 10 11:44:17 dist-test-slave-njxd krb5kdc[24054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:17.567539 24136 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } }
I20260810 11:44:17.568046 24136 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:17.568183 24136 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc45120139164e6d98434464ff77075f, State: Initialized, Role: FOLLOWER
I20260810 11:44:17.568908 24136 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } }
I20260810 11:44:17.569231 24136 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:17.569237 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:17.544248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.20.93.190:32863)
0810 11:44:17.545170 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:17.545188 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:17.545250 (+ 62us) server_negotiation.cc:407] Connection header received
0810 11:44:17.547789 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:17.547822 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:17.548380 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:17.548895 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:17.550522 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:17.552614 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:17.554289 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:17.555254 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:17.555458 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:17.559841 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:17.559857 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:17.559881 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:17.559933 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:17.564216 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:17.565052 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:17.565062 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:17.565074 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:17.565337 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:17.565781 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:17.565787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:17.565790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:17.566634 (+ 844us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:17.566947 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:17.567290 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 11:44:17.567797 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":370,"threads_started":1}
I20260810 11:44:17.569408 24136 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:17.569648 24136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:17.572269 24136 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } }
I20260810 11:44:17.572857 24136 leader_election.cc:302] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc45120139164e6d98434464ff77075f; no voters:
I20260810 11:44:17.573415 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:44:17.573534 24136 leader_election.cc:288] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:17.573684 24142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:17.576356 24136 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:17.577885 24142 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [term 1 LEADER]: Becoming Leader. State: Replica: fc45120139164e6d98434464ff77075f, State: Running, Role: LEADER
I20260810 11:44:17.578665 24142 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } }
I20260810 11:44:17.582823 24143 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc45120139164e6d98434464ff77075f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } } }
I20260810 11:44:17.583215 24143 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:17.582791 24144 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc45120139164e6d98434464ff77075f. Latest consensus state: current_term: 1 leader_uuid: "fc45120139164e6d98434464ff77075f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc45120139164e6d98434464ff77075f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 32863 } } }
I20260810 11:44:17.583843 24144 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:17.584851 24151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:17.590781 24151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:17.607003 24151 catalog_manager.cc:1347] Generated new cluster ID: df0a917288fa46419b5d5e3c9f30b2f0
I20260810 11:44:17.607164 24151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:17.620810 24164 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:44:17.627161 24151 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:17.628949 24151 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:17.635485 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:17.624391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.20.93.190:32863)
0810 11:44:17.624604 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:17.624613 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:17.626221 (+ 1608us) server_negotiation.cc:407] Connection header received
0810 11:44:17.626534 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:17.626540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:17.626624 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:17.626754 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:17.630044 (+ 3290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:17.630794 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:17.632092 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:17.632530 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:17.632678 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:17.633907 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:17.633913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:17.633916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:17.634598 (+ 682us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:17.634651 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:17.634676 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:17.634799 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:17.634952 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 11:44:17.635085 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:44:17.643198 24151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc45120139164e6d98434464ff77075f: Generated new TSK 0
I20260810 11:44:17.644344 24151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:44:17.656642 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24067
I20260810 11:44:17.657812 24131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:17.749048 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24067
2026-08-10T11:44:17Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2067 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:17 dist-test-slave-njxd krb5kdc[24173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:17 dist-test-slave-njxd krb5kdc[24173](info): set up 2 sockets
Aug 10 11:44:17 dist-test-slave-njxd krb5kdc[24173](info): commencing operation
krb5kdc: starting...
W20260810 11:44:18.884209 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:19 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:19.121383 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:39495
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:39495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:19.460638 24187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:19.460947 24187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:19.461066 24187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:19.468472 24187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:19.468590 24187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:19.468639 24187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:19.468725 24187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:19.468815 24187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:19.478720 24187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:39495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:39495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:19.480103 24187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:19.481441 24187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:19.493465 24196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:19.493427 24193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:19.493495 24192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:19.494395 24187 server_base.cc:1034] running on GCE node
I20260810 11:44:19.495247 24187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:19.496560 24187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:19.497761 24187 hybrid_clock.cc:648] HybridClock initialized: now 1786362259497719 us; error 30 us; skew 500 ppm
I20260810 11:44:19.498070 24187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:19 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362259, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:19.501646 24187 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:19.503062 24187 webserver.cc:460] Webserver started at http://127.20.93.190:39835/ using document root <none> and password file <none>
I20260810 11:44:19.503705 24187 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:19.503810 24187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:19.504061 24187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:19.507159 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a77115b4acba4b659518fc4c91df02c8"
format_stamp: "Formatted at 2026-08-10 11:44:19 on dist-test-slave-njxd"
server_key: "af9891c0a299d3aab7a564e4ab578509"
server_key_iv: "dd09829e6318f1455a617f1c727ea2b1"
server_key_version: "encryptionkey@0"
I20260810 11:44:19.507889 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a77115b4acba4b659518fc4c91df02c8"
format_stamp: "Formatted at 2026-08-10 11:44:19 on dist-test-slave-njxd"
server_key: "af9891c0a299d3aab7a564e4ab578509"
server_key_iv: "dd09829e6318f1455a617f1c727ea2b1"
server_key_version: "encryptionkey@0"
I20260810 11:44:19.513219 24187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:44:19.517001 24204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:19.518491 24187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:19.518694 24187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a77115b4acba4b659518fc4c91df02c8"
format_stamp: "Formatted at 2026-08-10 11:44:19 on dist-test-slave-njxd"
server_key: "af9891c0a299d3aab7a564e4ab578509"
server_key_iv: "dd09829e6318f1455a617f1c727ea2b1"
server_key_version: "encryptionkey@0"
I20260810 11:44:19.518910 24187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:19.552649 24187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:19.556846 24187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:19.557156 24187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:19.576728 24187 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:39495
I20260810 11:44:19.576752 24256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:39495 every 8 connection(s)
I20260810 11:44:19.578286 24187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:19.582124 24257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:19.586833 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24187
I20260810 11:44:19.587059 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:19.587742 20854 external_mini_cluster.cc:1428] Setting key 85b2bbea88b3f9809d8f4ece817daf23
I20260810 11:44:19.594010 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8: Bootstrap starting.
I20260810 11:44:19.598155 24257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:19.599304 24257 log.cc:824] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:19.602641 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8: No bootstrap required, opened a new log
Aug 10 11:44:19 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:19.611335 24257 raft_consensus.cc:348] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } }
I20260810 11:44:19.611748 24257 raft_consensus.cc:374] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:19.611894 24257 raft_consensus.cc:729] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77115b4acba4b659518fc4c91df02c8, State: Initialized, Role: FOLLOWER
I20260810 11:44:19.612653 24257 consensus_queue.cc:260] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } }
I20260810 11:44:19.612948 24257 raft_consensus.cc:388] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:19.613091 24257 raft_consensus.cc:482] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:19.613296 24257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:19.614456 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:19.590174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.20.93.190:39495)
0810 11:44:19.591278 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:19.591301 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:19.591759 (+ 458us) server_negotiation.cc:407] Connection header received
0810 11:44:19.594175 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:19.594207 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:19.594759 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:19.595192 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:19.597687 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:19.599096 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:19.600677 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:19.601348 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:19.601486 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:19.606338 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:19.606347 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:19.606362 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:19.606407 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:19.609642 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:19.610377 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:19.610388 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:19.610401 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:19.610593 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:19.611010 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:19.611015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:19.611020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:19.611580 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:19.611962 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:19.612345 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:44:19.612834 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":503,"threads_started":1}
I20260810 11:44:19.616189 24257 raft_consensus.cc:504] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } }
I20260810 11:44:19.616726 24257 leader_election.cc:302] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a77115b4acba4b659518fc4c91df02c8; no voters:
I20260810 11:44:19.617420 24257 leader_election.cc:288] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:19.617607 24262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:19.618396 24262 raft_consensus.cc:686] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [term 1 LEADER]: Becoming Leader. State: Replica: a77115b4acba4b659518fc4c91df02c8, State: Running, Role: LEADER
I20260810 11:44:19.619127 24262 consensus_queue.cc:237] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } }
I20260810 11:44:19.620025 24257 sys_catalog.cc:564] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:19.622433 24264 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a77115b4acba4b659518fc4c91df02c8. Latest consensus state: current_term: 1 leader_uuid: "a77115b4acba4b659518fc4c91df02c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } } }
I20260810 11:44:19.622445 24263 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a77115b4acba4b659518fc4c91df02c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77115b4acba4b659518fc4c91df02c8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39495 } } }
I20260810 11:44:19.623256 24263 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:19.624321 24264 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:19.626634 24268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:19.632001 24268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:19.642958 24268 catalog_manager.cc:1347] Generated new cluster ID: 81c7389bc15b47f4aa925669c9132a1d
I20260810 11:44:19.643085 24268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:19.657991 24268 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:19.659358 24268 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:19.671681 24268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a77115b4acba4b659518fc4c91df02c8: Generated new TSK 0
I20260810 11:44:19.672717 24268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:19.730551 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:20.091737 24283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:20.092100 24283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:20.092231 24283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:20.099758 24283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:20.099881 24283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:20.100000 24283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:20.110348 24283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:20.111668 24283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:20.113095 24283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:20.127460 24288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:20.127553 24289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:20.127777 24292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:20.128629 24283 server_base.cc:1034] running on GCE node
I20260810 11:44:20.129307 24283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:20.130436 24283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:20.131659 24283 hybrid_clock.cc:648] HybridClock initialized: now 1786362260131617 us; error 43 us; skew 500 ppm
I20260810 11:44:20.131958 24283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:20 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362260, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:20.135267 24283 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:20.136723 24283 webserver.cc:460] Webserver started at http://127.20.93.129:43697/ using document root <none> and password file <none>
I20260810 11:44:20.137372 24283 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:20.137476 24283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:20.137814 24283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:20.140671 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "2939ab791c704f24aba309e441564e02"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "50e80874340608a9b13ebae392e97088"
server_key_iv: "627a0b67bb8392336567c42156a52871"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.141470 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "2939ab791c704f24aba309e441564e02"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "50e80874340608a9b13ebae392e97088"
server_key_iv: "627a0b67bb8392336567c42156a52871"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.146858 24283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:44:20.150614 24300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.152019 24283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:44:20.152205 24283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "2939ab791c704f24aba309e441564e02"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "50e80874340608a9b13ebae392e97088"
server_key_iv: "627a0b67bb8392336567c42156a52871"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.152420 24283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:20.168301 24283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:20.173221 24283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:20.173542 24283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:20.174741 24283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:20.176141 24283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:20.176246 24283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.176399 24283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:20.176471 24283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.211463 24283 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:32913
I20260810 11:44:20.211505 24421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:32913 every 8 connection(s)
I20260810 11:44:20.212932 24283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:20.214763 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24283
I20260810 11:44:20.214988 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:20.215791 20854 external_mini_cluster.cc:1428] Setting key 7ac2225e1e2c22839b1490c9b8c35aa2
Aug 10 11:44:20 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362260, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:20.237056 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:20.216111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:47047 (local address 127.20.93.190:39495)
0810 11:44:20.216614 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:20.216620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:20.218159 (+ 1539us) server_negotiation.cc:407] Connection header received
0810 11:44:20.219802 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:20.219808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:20.219896 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:20.220027 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:20.221798 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.222725 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.224446 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.225004 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.225159 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:20.229525 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:20.229534 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:20.229552 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:20.229627 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:20.233525 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:20.234408 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:20.234417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:20.234424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:20.234605 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:20.235067 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:20.235075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:20.235081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:20.235496 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:20.235671 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:20.236436 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 11:44:20.236684 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
I20260810 11:44:20.238209 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:20.216742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:39495 (local address 127.20.93.129:47047)
0810 11:44:20.217678 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:20.217726 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:20.219505 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:20.220348 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:20.220358 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:20.220903 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:20.221610 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:20.221647 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.222911 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.222915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:20.224290 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:20.224302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.225257 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.225267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:20.225649 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:20.226421 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:20.226453 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:20.229266 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:20.233740 (+ 4474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:20.233748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:20.233764 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:20.234256 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:20.234727 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:20.234730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:20.234733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:20.234932 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:20.235635 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:20.235641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:20.236018 (+ 377us) client_negotiation.cc:769] Sending connection context
0810 11:44:20.236317 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 11:44:20.236927 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":390,"threads_started":1}
I20260810 11:44:20.240020 24422 heartbeater.cc:347] Connected to a master server at 127.20.93.190:39495
I20260810 11:44:20.240367 24422 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:44:20.241269 24422 heartbeater.cc:511] Master 127.20.93.190:39495 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:44:20.243575 24222 ts_manager.cc:194] Registered new tserver with Master: 2939ab791c704f24aba309e441564e02 (127.20.93.129:32913)
I20260810 11:44:20.245133 24222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:47047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:20.272987 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:20.614004 24428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:20.614288 24428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:20.614382 24428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:20.621738 24428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:20.621855 24428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:20.622035 24428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:20.632306 24428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:20.633659 24428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:20.635040 24428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:20.648442 24435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:20.648398 24437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:20.648476 24434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:20.649415 24428 server_base.cc:1034] running on GCE node
I20260810 11:44:20.650193 24428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:20.651163 24428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:20.652365 24428 hybrid_clock.cc:648] HybridClock initialized: now 1786362260652323 us; error 41 us; skew 500 ppm
I20260810 11:44:20.652698 24428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:20 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362260, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:20.656049 24428 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:20.657671 24428 webserver.cc:460] Webserver started at http://127.20.93.130:43981/ using document root <none> and password file <none>
I20260810 11:44:20.658411 24428 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:20.658562 24428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:20.658890 24428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:20.661944 24428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "34eb7339f9994f83bce335c28f649f1c"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "5e7b359b2a855a733b4e78134b88fe5a"
server_key_iv: "37c0bc2408783cccf0dbb4c655fe12b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.662746 24428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "34eb7339f9994f83bce335c28f649f1c"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "5e7b359b2a855a733b4e78134b88fe5a"
server_key_iv: "37c0bc2408783cccf0dbb4c655fe12b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.667979 24428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:20.671370 24445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.672809 24428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:20.672979 24428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "34eb7339f9994f83bce335c28f649f1c"
format_stamp: "Formatted at 2026-08-10 11:44:20 on dist-test-slave-njxd"
server_key: "5e7b359b2a855a733b4e78134b88fe5a"
server_key_iv: "37c0bc2408783cccf0dbb4c655fe12b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:20.673184 24428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:20.695690 24428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:20.699779 24428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:20.700093 24428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:20.701112 24428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:20.702553 24428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:20.702682 24428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.702826 24428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:20.702901 24428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:20.737464 24428 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:41267
I20260810 11:44:20.737500 24566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:41267 every 8 connection(s)
I20260810 11:44:20.738873 24428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:20.747761 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24428
I20260810 11:44:20.747967 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:20.748437 20854 external_mini_cluster.cc:1428] Setting key 74511fb100af70591164523961a2d470
Aug 10 11:44:20 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362260, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:20.760351 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:20.741741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:52059 (local address 127.20.93.190:39495)
0810 11:44:20.742220 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:20.742226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:20.743296 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 11:44:20.745100 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:20.745104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:20.745174 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:20.745283 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:20.746910 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.747657 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.749564 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.750338 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.750460 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:20.754168 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:20.754173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:20.754176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:20.754210 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:20.756980 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:20.757936 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:20.757940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:20.757943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:20.758076 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:20.758537 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:20.758541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:20.758543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:20.758809 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:20.758970 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:20.759693 (+ 723us) server_negotiation.cc:299] Negotiation successful
0810 11:44:20.759939 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":341,"thread_start_us":218,"threads_started":1}
I20260810 11:44:20.761339 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:20.742393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:39495 (local address 127.20.93.130:52059)
0810 11:44:20.743113 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:20.743158 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:20.744831 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:20.745507 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:20.745515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:20.746130 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:20.746740 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:20.746758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.747842 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.747846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:20.749381 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:20.749396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:20.750621 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:20.750628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:20.750859 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:20.751578 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:20.751603 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:20.753918 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:20.757128 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:20.757138 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:20.757190 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:20.757798 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:20.758202 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:20.758207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:20.758210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:20.758405 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:20.758954 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:20.758962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:20.759315 (+ 353us) client_negotiation.cc:769] Sending connection context
0810 11:44:20.759584 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 11:44:20.760157 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":275,"threads_started":1}
I20260810 11:44:20.762929 24567 heartbeater.cc:347] Connected to a master server at 127.20.93.190:39495
I20260810 11:44:20.763301 24567 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:20.764271 24567 heartbeater.cc:511] Master 127.20.93.190:39495 requested a full tablet report, sending...
I20260810 11:44:20.766054 24222 ts_manager.cc:194] Registered new tserver with Master: 34eb7339f9994f83bce335c28f649f1c (127.20.93.130:41267)
I20260810 11:44:20.767378 24222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:52059
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:20.801131 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:21.138268 24574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:21.138554 24574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:21.138650 24574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:21.145905 24574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:21.146008 24574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:21.146118 24574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:21.156548 24574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:39495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:21.157976 24574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:21.159277 24574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:21.172600 24583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:21.172649 24580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:21.172551 24579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:21.177786 24574 server_base.cc:1034] running on GCE node
I20260810 11:44:21.178499 24574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:21.179450 24574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:21.180646 24574 hybrid_clock.cc:648] HybridClock initialized: now 1786362261180601 us; error 37 us; skew 500 ppm
I20260810 11:44:21.180979 24574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:21.184149 24574 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:21.185753 24574 webserver.cc:460] Webserver started at http://127.20.93.131:42521/ using document root <none> and password file <none>
I20260810 11:44:21.186472 24574 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:21.186604 24574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:21.186933 24574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:21.189872 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "f872059f986c4697883db3f03a73dfc1"
format_stamp: "Formatted at 2026-08-10 11:44:21 on dist-test-slave-njxd"
server_key: "388029f2255cbc3a1636fcfbcbbc404b"
server_key_iv: "dc1157e0269a8ec3412561a0cea66979"
server_key_version: "encryptionkey@0"
I20260810 11:44:21.190688 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "f872059f986c4697883db3f03a73dfc1"
format_stamp: "Formatted at 2026-08-10 11:44:21 on dist-test-slave-njxd"
server_key: "388029f2255cbc3a1636fcfbcbbc404b"
server_key_iv: "dc1157e0269a8ec3412561a0cea66979"
server_key_version: "encryptionkey@0"
I20260810 11:44:21.195879 24574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:21.199282 24590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:21.200595 24574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:21.200780 24574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "f872059f986c4697883db3f03a73dfc1"
format_stamp: "Formatted at 2026-08-10 11:44:21 on dist-test-slave-njxd"
server_key: "388029f2255cbc3a1636fcfbcbbc404b"
server_key_iv: "dc1157e0269a8ec3412561a0cea66979"
server_key_version: "encryptionkey@0"
I20260810 11:44:21.200992 24574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:21.219426 24574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:21.223989 24574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:21.224294 24574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:21.225318 24574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:21.226819 24574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:21.226919 24574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:21.227092 24574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:21.227190 24574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:21.248368 24422 heartbeater.cc:503] Master 127.20.93.190:39495 was elected leader, sending a full tablet report...
I20260810 11:44:21.261396 24574 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:36611
I20260810 11:44:21.261445 24709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:36611 every 8 connection(s)
I20260810 11:44:21.262764 24574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:21.263727 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24574
I20260810 11:44:21.263906 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:21.264505 20854 external_mini_cluster.cc:1428] Setting key 12aa03d80f7696103c1cd6d1e1966a61
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:21.283444 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.265542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:53091 (local address 127.20.93.190:39495)
0810 11:44:21.266112 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:21.266118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:21.267280 (+ 1162us) server_negotiation.cc:407] Connection header received
0810 11:44:21.269275 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:21.269278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:21.269342 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:21.269430 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:21.270992 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.271757 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.273288 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.273747 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.273841 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.277259 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:21.277264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:21.277267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:21.277298 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:21.280786 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.281435 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.281439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.281441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.281557 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.281991 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.281994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.281996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.282253 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:21.282365 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:21.283013 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 11:44:21.283172 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":268,"threads_started":1}
I20260810 11:44:21.284394 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.266245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:39495 (local address 127.20.93.131:53091)
0810 11:44:21.267091 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:21.267141 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:21.269027 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:21.269716 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:21.269725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:21.270254 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:21.270838 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:21.270855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.271887 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.271890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:21.273142 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:21.273155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.273935 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.273940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:21.274136 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.274770 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:21.274794 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:21.277061 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:21.280906 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:21.280912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:21.280926 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:21.281335 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:21.281704 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:21.281707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:21.281709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:21.281888 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:21.282365 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:21.282371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:21.282692 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 11:44:21.282914 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 11:44:21.283299 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":336,"threads_started":1}
I20260810 11:44:21.285816 24711 heartbeater.cc:347] Connected to a master server at 127.20.93.190:39495
I20260810 11:44:21.286134 24711 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:21.286973 24711 heartbeater.cc:511] Master 127.20.93.190:39495 requested a full tablet report, sending...
I20260810 11:44:21.288409 24222 ts_manager.cc:194] Registered new tserver with Master: f872059f986c4697883db3f03a73dfc1 (127.20.93.131:36611)
I20260810 11:44:21.289152 24222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:53091
I20260810 11:44:21.292584 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:21.347150 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.332980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.20.93.190:39495)
0810 11:44:21.333191 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:21.333197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:21.333802 (+ 605us) server_negotiation.cc:407] Connection header received
0810 11:44:21.333963 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:21.333968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:21.334045 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:21.334133 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:21.335548 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.336282 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.337558 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.338050 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.338195 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.342112 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:21.342116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:21.342120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:21.342155 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:21.344732 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.345395 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.345399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.345401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.345526 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.345921 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.345924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.345926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.346259 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:21.346454 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:21.346756 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 11:44:21.346926 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 11:44:21.347786 24222 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49644
I20260810 11:44:21.349614 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24283
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:21.364920 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.350599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.20.93.129:32913)
0810 11:44:21.351227 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:21.351234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:21.351258 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:21.351315 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:21.351321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:21.351532 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:21.351657 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:21.352813 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.353921 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.355129 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.355721 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.355880 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.359811 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:21.359822 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:21.359836 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:21.359875 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:21.362590 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.363228 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.363236 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.363238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.363359 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.363665 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.363669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.363670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.363990 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:21.364214 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:21.364414 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 11:44:21.364628 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":316,"threads_started":1}
W20260810 11:44:21.366271 24417 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41004
W20260810 11:44:21.367443 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:21.367666 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24283
I20260810 11:44:21.387152 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24428
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:21.402827 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.388046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.20.93.130:41267)
0810 11:44:21.388696 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:21.388704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:21.388730 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:44:21.388792 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:21.388798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:21.389023 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:21.389160 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:21.390476 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.391484 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.392706 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.393259 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.393361 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.397455 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:21.397466 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:21.397481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:21.397522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:21.400288 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.400891 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.400897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.400900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.401026 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.401345 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.401348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.401350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.401837 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:21.402077 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:21.402313 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 11:44:21.402527 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":350,"threads_started":1}
W20260810 11:44:21.404094 24562 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32796
W20260810 11:44:21.405308 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24428: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:21.405496 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24428
I20260810 11:44:21.423890 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24574
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362261, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:21.440541 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:21.424910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.20.93.131:36611)
0810 11:44:21.425689 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:21.425698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:21.425731 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:44:21.425797 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:21.425804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:21.426014 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:21.426159 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:21.427631 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.428690 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.429987 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:21.430542 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:21.430753 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:21.435173 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:21.435184 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:21.435208 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:21.435245 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:21.437994 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.438634 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.438641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.438643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.438765 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:21.439081 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:21.439084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:21.439086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:21.439444 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:21.439672 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:21.439921 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:44:21.440130 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":340,"threads_started":1}
W20260810 11:44:21.441980 24704 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52440
W20260810 11:44:21.443133 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24574: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:44:21.443302 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24574
I20260810 11:44:21.461055 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24187
I20260810 11:44:21.462352 24252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:21.567538 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24187
2026-08-10T11:44:21Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3824 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24730](info): set up 2 sockets
Aug 10 11:44:21 dist-test-slave-njxd krb5kdc[24730](info): commencing operation
krb5kdc: starting...
W20260810 11:44:22.721393 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:22 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:22.957930 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35223
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:23.305344 24743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:23.305732 24743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:23.305858 24743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:23.312996 24743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:23.313097 24743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:23.313143 24743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:23.313201 24743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:23.313246 24743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:23.322909 24743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:23.324151 24743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:23.325403 24743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:23.337524 24750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:23.337700 24751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:23.337600 24754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:23.338317 24743 server_base.cc:1034] running on GCE node
I20260810 11:44:23.339169 24743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:23.340553 24743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:23.341745 24743 hybrid_clock.cc:648] HybridClock initialized: now 1786362263341702 us; error 29 us; skew 500 ppm
I20260810 11:44:23.342103 24743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:23 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362263, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:23.345299 24743 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:23.346817 24743 webserver.cc:460] Webserver started at http://127.20.93.190:38693/ using document root <none> and password file <none>
I20260810 11:44:23.347530 24743 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:23.347664 24743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:23.348001 24743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:23.351167 24743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "def252596f574d4799e836d76c5ff410"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "166712380eef4b1459c289b6c800769e"
server_key_iv: "98b6eb91131a8439be06b36b2354c4d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.351996 24743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "def252596f574d4799e836d76c5ff410"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "166712380eef4b1459c289b6c800769e"
server_key_iv: "98b6eb91131a8439be06b36b2354c4d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.357158 24743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 11:44:23.360808 24761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:23.362251 24743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:23.362439 24743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "def252596f574d4799e836d76c5ff410"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "166712380eef4b1459c289b6c800769e"
server_key_iv: "98b6eb91131a8439be06b36b2354c4d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.362644 24743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:23.394613 24743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:23.398895 24743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:23.399194 24743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:23.418919 24743 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35223
I20260810 11:44:23.418967 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35223 every 8 connection(s)
I20260810 11:44:23.420380 24743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:23.424396 24816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:23.430529 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24743
I20260810 11:44:23.430804 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:23.431664 20854 external_mini_cluster.cc:1428] Setting key 3c4d381224c5613e73e8a39ce22a5cb4
I20260810 11:44:23.436131 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410: Bootstrap starting.
I20260810 11:44:23.440038 24816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:23.441222 24816 log.cc:824] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:23.444523 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410: No bootstrap required, opened a new log
Aug 10 11:44:23 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:23.452994 24816 raft_consensus.cc:348] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } }
I20260810 11:44:23.453404 24816 raft_consensus.cc:374] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:23.453527 24816 raft_consensus.cc:729] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: def252596f574d4799e836d76c5ff410, State: Initialized, Role: FOLLOWER
I20260810 11:44:23.454272 24816 consensus_queue.cc:260] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } }
I20260810 11:44:23.454563 24816 raft_consensus.cc:388] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:23.454717 24816 raft_consensus.cc:482] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:23.454916 24816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:23.456282 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:23.433976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.20.93.190:35223)
0810 11:44:23.434841 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:23.434854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:23.434896 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:44:23.436683 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:23.436706 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:23.437104 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:23.437439 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:23.439442 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:23.440850 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:23.442508 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:23.443195 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:23.443348 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:23.448293 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:23.448301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:23.448317 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:23.448356 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:23.451686 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:23.452440 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:23.452447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:23.452454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:23.452623 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:23.453036 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:23.453040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:23.453042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:23.453776 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:23.454107 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:23.454593 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 11:44:23.455229 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":411,"threads_started":1}
I20260810 11:44:23.457376 24816 raft_consensus.cc:504] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } }
I20260810 11:44:23.457927 24816 leader_election.cc:302] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: def252596f574d4799e836d76c5ff410; no voters:
I20260810 11:44:23.458654 24816 leader_election.cc:288] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:23.458849 24821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:23.461280 24816 sys_catalog.cc:564] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:23.461705 24821 raft_consensus.cc:686] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [term 1 LEADER]: Becoming Leader. State: Replica: def252596f574d4799e836d76c5ff410, State: Running, Role: LEADER
I20260810 11:44:23.462286 24821 consensus_queue.cc:237] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } }
I20260810 11:44:23.465612 24825 sys_catalog.cc:455] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [sys.catalog]: SysCatalogTable state changed. Reason: New leader def252596f574d4799e836d76c5ff410. Latest consensus state: current_term: 1 leader_uuid: "def252596f574d4799e836d76c5ff410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } } }
I20260810 11:44:23.465740 24822 sys_catalog.cc:455] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "def252596f574d4799e836d76c5ff410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def252596f574d4799e836d76c5ff410" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35223 } } }
I20260810 11:44:23.466156 24822 sys_catalog.cc:458] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:23.466171 24825 sys_catalog.cc:458] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:23.472748 24830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:23.477245 24830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:23.486367 24830 catalog_manager.cc:1347] Generated new cluster ID: 95f339aa6ece45c5801cf877e577a347
I20260810 11:44:23.486505 24830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:23.505803 24830 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:23.507189 24830 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:23.514889 24830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P def252596f574d4799e836d76c5ff410: Generated new TSK 0
I20260810 11:44:23.515734 24830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:23.578265 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:23.926335 24846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:23.926656 24846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:23.926834 24846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:23.935711 24846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:23.935853 24846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:23.936020 24846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:23.945950 24846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:23.947284 24846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:23.948648 24846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:23.962785 24855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:23.962755 24853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:23.962837 24851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:23.963719 24846 server_base.cc:1034] running on GCE node
I20260810 11:44:23.964406 24846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:23.965447 24846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:23.966679 24846 hybrid_clock.cc:648] HybridClock initialized: now 1786362263966648 us; error 82 us; skew 500 ppm
I20260810 11:44:23.967078 24846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:23 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362263, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:23.970469 24846 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:23.972092 24846 webserver.cc:460] Webserver started at http://127.20.93.129:46805/ using document root <none> and password file <none>
I20260810 11:44:23.972828 24846 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:23.972980 24846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:23.973335 24846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:23.976488 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "32160fc05f304571a4d2d1998b7fc375"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "b13009a75de229a537dfb03f1bb2a507"
server_key_iv: "3d24260b9e43dab7bf8b6fa3462b082f"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.977310 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "32160fc05f304571a4d2d1998b7fc375"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "b13009a75de229a537dfb03f1bb2a507"
server_key_iv: "3d24260b9e43dab7bf8b6fa3462b082f"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.982944 24846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:44:23.987089 24864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:23.988580 24846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 11:44:23.988780 24846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "32160fc05f304571a4d2d1998b7fc375"
format_stamp: "Formatted at 2026-08-10 11:44:23 on dist-test-slave-njxd"
server_key: "b13009a75de229a537dfb03f1bb2a507"
server_key_iv: "3d24260b9e43dab7bf8b6fa3462b082f"
server_key_version: "encryptionkey@0"
I20260810 11:44:23.989006 24846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:24.006417 24846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:24.010603 24846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:24.010927 24846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:24.011927 24846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:24.013348 24846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:24.013458 24846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:24.013653 24846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:24.013737 24846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:24.048367 24846 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:39403
I20260810 11:44:24.048427 24984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:39403 every 8 connection(s)
I20260810 11:44:24.049865 24846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:24.053469 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24846
I20260810 11:44:24.053767 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:24.054661 20854 external_mini_cluster.cc:1428] Setting key 9b1a238d77c8038f1df59a1531988f2d
Aug 10 11:44:24 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362263, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:24.072479 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:24.052459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:34559 (local address 127.20.93.190:35223)
0810 11:44:24.052884 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:24.052889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:24.054264 (+ 1375us) server_negotiation.cc:407] Connection header received
0810 11:44:24.056187 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:24.056192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:24.056287 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:24.056408 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:24.057997 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.058719 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.060275 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.060702 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.060837 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:24.064775 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:24.064780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:24.064784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:24.064816 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:24.068582 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:24.069429 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:24.069437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:24.069443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:24.069642 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:24.070213 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:24.070221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:24.070227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:24.070665 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:24.070827 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:24.071680 (+ 853us) server_negotiation.cc:299] Negotiation successful
0810 11:44:24.071971 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":205,"threads_started":1}
I20260810 11:44:24.073288 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:24.053107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35223 (local address 127.20.93.129:34559)
0810 11:44:24.054099 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:24.054137 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:24.055919 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:24.056655 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:24.056665 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:24.057182 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:24.057825 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:24.057845 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.058892 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.058896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:24.060116 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:24.060129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.060925 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.060933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:24.061203 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:24.061971 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:24.061994 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:24.064532 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:24.068742 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:24.068750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:24.068766 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:24.069279 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:24.069801 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:24.069806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:24.069810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:24.070030 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:24.070793 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:24.070800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:24.071245 (+ 445us) client_negotiation.cc:769] Sending connection context
0810 11:44:24.071531 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 11:44:24.071973 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":461,"threads_started":1}
I20260810 11:44:24.075106 24985 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35223
I20260810 11:44:24.075510 24985 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:24.076634 24985 heartbeater.cc:511] Master 127.20.93.190:35223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:44:24.079150 24780 ts_manager.cc:194] Registered new tserver with Master: 32160fc05f304571a4d2d1998b7fc375 (127.20.93.129:39403)
I20260810 11:44:24.080681 24780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:34559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:24.109099 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:24.449178 24991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:24.449550 24991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:24.449766 24991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:24.457279 24991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:24.457459 24991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:24.457744 24991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:24.468765 24991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:24.470355 24991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:24.471827 24991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:24.486109 24999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:24.486195 24997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:24.486408 24996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:24.486789 24991 server_base.cc:1034] running on GCE node
I20260810 11:44:24.487505 24991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:24.488477 24991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:24.489697 24991 hybrid_clock.cc:648] HybridClock initialized: now 1786362264489633 us; error 42 us; skew 500 ppm
I20260810 11:44:24.490033 24991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:24 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362264, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:24.493294 24991 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:24.494951 24991 webserver.cc:460] Webserver started at http://127.20.93.130:45295/ using document root <none> and password file <none>
I20260810 11:44:24.495704 24991 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:24.495863 24991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:24.496201 24991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:24.499378 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "b33da202e924406a8c4ec5122b6e3da5"
format_stamp: "Formatted at 2026-08-10 11:44:24 on dist-test-slave-njxd"
server_key: "fb2835e79f8143007377a0b8ff4621f1"
server_key_iv: "566ba2c7ab5b9f6a236ac7e9d232dcff"
server_key_version: "encryptionkey@0"
I20260810 11:44:24.500320 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "b33da202e924406a8c4ec5122b6e3da5"
format_stamp: "Formatted at 2026-08-10 11:44:24 on dist-test-slave-njxd"
server_key: "fb2835e79f8143007377a0b8ff4621f1"
server_key_iv: "566ba2c7ab5b9f6a236ac7e9d232dcff"
server_key_version: "encryptionkey@0"
I20260810 11:44:24.506116 24991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:44:24.509832 25006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:24.511330 24991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:24.511516 24991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "b33da202e924406a8c4ec5122b6e3da5"
format_stamp: "Formatted at 2026-08-10 11:44:24 on dist-test-slave-njxd"
server_key: "fb2835e79f8143007377a0b8ff4621f1"
server_key_iv: "566ba2c7ab5b9f6a236ac7e9d232dcff"
server_key_version: "encryptionkey@0"
I20260810 11:44:24.511742 24991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:24.527199 24991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:24.531386 24991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:24.531697 24991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:24.532723 24991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:24.534292 24991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:24.534366 24991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:24.534456 24991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:24.534499 24991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:24.568441 24991 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:36479
I20260810 11:44:24.568483 25123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:36479 every 8 connection(s)
I20260810 11:44:24.570000 24991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:24.573001 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 24991
I20260810 11:44:24.573205 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:24.573781 20854 external_mini_cluster.cc:1428] Setting key d1021fcdb5ab692a595d8a92d56c0bdb
Aug 10 11:44:24 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362264, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:24.593218 25126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:24.572718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:41793 (local address 127.20.93.190:35223)
0810 11:44:24.573362 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:24.573367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:24.574665 (+ 1298us) server_negotiation.cc:407] Connection header received
0810 11:44:24.576780 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:24.576788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:24.576883 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:24.577011 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:24.578673 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.579889 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.582208 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.583023 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.583185 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:24.587052 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:24.587058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:24.587062 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:24.587098 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:24.589872 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:24.590659 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:24.590664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:24.590667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:24.590798 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:24.591252 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:24.591261 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:24.591263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:24.591522 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:24.591715 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:24.592459 (+ 744us) server_negotiation.cc:299] Negotiation successful
0810 11:44:24.592696 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":494,"thread_start_us":323,"threads_started":1}
I20260810 11:44:24.594303 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:24.573429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35223 (local address 127.20.93.130:41793)
0810 11:44:24.574440 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:24.574492 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:24.576462 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:24.577240 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:24.577249 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:24.577872 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:24.578485 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:24.578505 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.580056 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.580063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:24.582035 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:24.582052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:24.583269 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:24.583277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:24.583592 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:24.584418 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:24.584442 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:24.586782 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:24.590026 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:24.590036 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:24.590055 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:24.590510 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:24.590917 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:24.590922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:24.590926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:24.591126 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:24.591704 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:24.591713 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:24.592080 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 11:44:24.592361 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 11:44:24.592961 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":409,"threads_started":1}
I20260810 11:44:24.595988 25124 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35223
I20260810 11:44:24.596390 25124 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:24.597404 25124 heartbeater.cc:511] Master 127.20.93.190:35223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
I20260810 11:44:24.599097 24780 ts_manager.cc:194] Registered new tserver with Master: b33da202e924406a8c4ec5122b6e3da5 (127.20.93.130:36479)
I20260810 11:44:24.599960 24780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:41793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:24.628952 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:24.967612 25130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:24.967900 25130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:24.967993 25130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:24.975440 25130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:24.975555 25130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:24.975673 25130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:24.985364 25130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:24.986857 25130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:24.988101 25130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:25.002056 25136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:25.002148 25139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:25.002091 25137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:25.003096 25130 server_base.cc:1034] running on GCE node
I20260810 11:44:25.003791 25130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:25.004768 25130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:25.005996 25130 hybrid_clock.cc:648] HybridClock initialized: now 1786362265005957 us; error 51 us; skew 500 ppm
I20260810 11:44:25.006320 25130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:25.009706 25130 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:25.011413 25130 webserver.cc:460] Webserver started at http://127.20.93.131:33667/ using document root <none> and password file <none>
I20260810 11:44:25.012087 25130 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:25.012184 25130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:25.012532 25130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:25.015563 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "a163ba40df004ee0a0fb69c9bb3082fc"
format_stamp: "Formatted at 2026-08-10 11:44:25 on dist-test-slave-njxd"
server_key: "032b650d2d5f2d0a8516473709d4ca38"
server_key_iv: "7cbb3bf468fa41cb58f85540476eef23"
server_key_version: "encryptionkey@0"
I20260810 11:44:25.016408 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "a163ba40df004ee0a0fb69c9bb3082fc"
format_stamp: "Formatted at 2026-08-10 11:44:25 on dist-test-slave-njxd"
server_key: "032b650d2d5f2d0a8516473709d4ca38"
server_key_iv: "7cbb3bf468fa41cb58f85540476eef23"
server_key_version: "encryptionkey@0"
I20260810 11:44:25.021762 25130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 11:44:25.025238 25146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:25.026599 25130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:25.026808 25130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "a163ba40df004ee0a0fb69c9bb3082fc"
format_stamp: "Formatted at 2026-08-10 11:44:25 on dist-test-slave-njxd"
server_key: "032b650d2d5f2d0a8516473709d4ca38"
server_key_iv: "7cbb3bf468fa41cb58f85540476eef23"
server_key_version: "encryptionkey@0"
I20260810 11:44:25.027032 25130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:25.047223 25130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:25.051692 25130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:25.052064 25130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:25.053032 25130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:25.054363 25130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:25.054471 25130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:25.054637 25130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:25.054711 25130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:25.084039 24985 heartbeater.cc:503] Master 127.20.93.190:35223 was elected leader, sending a full tablet report...
I20260810 11:44:25.092108 25130 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41363
I20260810 11:44:25.092167 25268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41363 every 8 connection(s)
I20260810 11:44:25.093549 25130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:25.103065 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25130
I20260810 11:44:25.103258 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:25.103698 20854 external_mini_cluster.cc:1428] Setting key 29014f2707750720af3c6d1d23fee012
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:25.113148 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.096137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:55465 (local address 127.20.93.190:35223)
0810 11:44:25.096579 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.096584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.097550 (+ 966us) server_negotiation.cc:407] Connection header received
0810 11:44:25.099393 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.099399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.099480 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.099610 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.101143 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.102029 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.103723 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.104160 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.104308 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.107954 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.107959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.107963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.107996 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.110494 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.111175 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.111179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.111181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.111294 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.111649 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.111652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.111654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.111939 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.112054 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.112699 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.112863 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":196,"threads_started":1}
I20260810 11:44:25.114197 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.096689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35223 (local address 127.20.93.131:55465)
0810 11:44:25.097379 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:25.097420 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:25.099108 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:25.099863 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:25.099871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:25.100389 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:25.100956 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:25.100974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.102179 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.102184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:25.103573 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:25.103585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.104404 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.104411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:25.104649 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.105360 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:25.105387 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:25.107767 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:25.110634 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:25.110641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:25.110654 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:25.111078 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:25.111386 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:25.111389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:25.111391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:25.111563 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:25.112044 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:25.112050 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:25.112377 (+ 327us) client_negotiation.cc:769] Sending connection context
0810 11:44:25.112608 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 11:44:25.112999 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":268,"threads_started":1}
I20260810 11:44:25.115739 25269 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35223
I20260810 11:44:25.116060 25269 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:25.116851 25269 heartbeater.cc:511] Master 127.20.93.190:35223 requested a full tablet report, sending...
I20260810 11:44:25.118532 24780 ts_manager.cc:194] Registered new tserver with Master: a163ba40df004ee0a0fb69c9bb3082fc (127.20.93.131:41363)
I20260810 11:44:25.119371 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:44:25.119640 24780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:55465
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:25.174849 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.159675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.20.93.129:39403)
0810 11:44:25.160299 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.160306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.160344 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:44:25.160510 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.160516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.160745 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.160878 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.162426 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.163539 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.164695 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.165229 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.165332 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.169311 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.169322 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.169337 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.169374 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.172317 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.172915 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.172922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.172925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.173048 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.173352 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.173355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.173357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.173778 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.174015 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.174226 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.174426 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":309,"threads_started":1}
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:25.193460 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.179482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.20.93.190:35223)
0810 11:44:25.179711 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.179718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.179977 (+ 259us) server_negotiation.cc:407] Connection header received
0810 11:44:25.180170 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.180176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.180267 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.180375 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.181668 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.182412 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.183644 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.184068 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.184204 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.188358 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.188372 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.188376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.188409 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.190852 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.191490 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.191498 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.191501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.191627 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.191993 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.191999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.192002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.192319 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.192574 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.192892 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.193137 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 11:44:25.194036 24780 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48166
I20260810 11:44:25.195021 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24846
I20260810 11:44:25.206960 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.195950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.20.93.129:39403)
0810 11:44:25.196185 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.196191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.196408 (+ 217us) server_negotiation.cc:407] Connection header received
0810 11:44:25.196522 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.196526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.196587 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.196657 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.197792 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.198471 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.199590 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.200003 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.200100 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.202357 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.202362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.202365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.202392 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.204741 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.205323 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.205327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.205330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.205447 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.205810 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.205813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.205815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.206048 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.206224 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.206446 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.206607 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:44:25.207548 24979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:25.289691 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24846
I20260810 11:44:25.309010 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24991
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:25.325294 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.310041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.20.93.130:36479)
0810 11:44:25.310628 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.310635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.310669 (+ 34us) server_negotiation.cc:407] Connection header received
0810 11:44:25.310732 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.310738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.310967 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.311162 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.312308 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.313338 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.314794 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.315372 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.315529 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.319738 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.319750 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.319765 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.319800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.322633 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.323284 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.323292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.323294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.323425 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.323761 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.323764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.323766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.324115 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.324351 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.324688 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.324975 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":304,"threads_started":1}
I20260810 11:44:25.326634 25119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:25.410269 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24991
I20260810 11:44:25.428292 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25130
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[24730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:25.446666 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:25.429392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50202 (local address 127.20.93.131:41363)
0810 11:44:25.430207 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:25.430214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:25.430264 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:44:25.430344 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:25.430351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:25.430568 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:25.430765 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:25.432166 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.433906 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.435319 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:25.436021 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:25.436164 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:25.440575 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:25.440588 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:25.440604 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:25.440648 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:25.443597 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.444342 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.444349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.444352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.444485 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:25.444902 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:25.444907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:25.444910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:25.445283 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:25.445634 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:25.446006 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 11:44:25.446302 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":478,"threads_started":1}
I20260810 11:44:25.447973 25263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:25.532868 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25130
I20260810 11:44:25.552405 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 24743
I20260810 11:44:25.553766 24811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:25.659458 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 24743
2026-08-10T11:44:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4092 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[25291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[25291](info): set up 2 sockets
Aug 10 11:44:25 dist-test-slave-njxd krb5kdc[25291](info): commencing operation
krb5kdc: starting...
W20260810 11:44:26.813853 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:26 dist-test-slave-njxd krb5kdc[25291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:27.057513 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:45699
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:45699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:27.405884 25308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:27.406230 25308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:27.406409 25308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:27.414016 25308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:27.414135 25308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:27.414242 25308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:27.414338 25308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:27.414422 25308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:27.424876 25308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:45699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:27.426328 25308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:27.427819 25308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:27.440284 25314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:27.440415 25313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:27.440439 25316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:27.441395 25308 server_base.cc:1034] running on GCE node
I20260810 11:44:27.442353 25308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:27.443974 25308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:27.445227 25308 hybrid_clock.cc:648] HybridClock initialized: now 1786362267445197 us; error 60 us; skew 500 ppm
I20260810 11:44:27.445665 25308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:27 dist-test-slave-njxd krb5kdc[25291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362267, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:27.448899 25308 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:27.450227 25308 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:27.450374 25308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:27.450728 25308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:27.454027 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "3013c471fa8e479d916fc09c0fe871f5"
format_stamp: "Formatted at 2026-08-10 11:44:27 on dist-test-slave-njxd"
server_key: "83cc0a02a8f44d040c0c26ba4b0f3e50"
server_key_iv: "3449fbf70b0790aa909d21b27d9674d6"
server_key_version: "encryptionkey@0"
I20260810 11:44:27.454922 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "3013c471fa8e479d916fc09c0fe871f5"
format_stamp: "Formatted at 2026-08-10 11:44:27 on dist-test-slave-njxd"
server_key: "83cc0a02a8f44d040c0c26ba4b0f3e50"
server_key_iv: "3449fbf70b0790aa909d21b27d9674d6"
server_key_version: "encryptionkey@0"
I20260810 11:44:27.460686 25308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 11:44:27.464511 25322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:27.466034 25308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:27.466248 25308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "3013c471fa8e479d916fc09c0fe871f5"
format_stamp: "Formatted at 2026-08-10 11:44:27 on dist-test-slave-njxd"
server_key: "83cc0a02a8f44d040c0c26ba4b0f3e50"
server_key_iv: "3449fbf70b0790aa909d21b27d9674d6"
server_key_version: "encryptionkey@0"
I20260810 11:44:27.466493 25308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:27.504189 25308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:27.508754 25308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:27.509060 25308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:27.529207 25308 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:45699
I20260810 11:44:27.529227 25378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:45699 every 8 connection(s)
I20260810 11:44:27.530256 25308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:27.534202 25379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:27.535006 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25308
I20260810 11:44:27.535250 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:27.535902 20854 external_mini_cluster.cc:1428] Setting key a9e6202882de672e26260c906125147a
I20260810 11:44:27.549559 25379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: Bootstrap starting.
Aug 10 11:44:27 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:27.553375 25379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:27.554922 25379 log.cc:824] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:27.564755 25379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: No bootstrap required, opened a new log
I20260810 11:44:27.567823 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:27.538112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.20.93.190:45699)
0810 11:44:27.538993 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:27.539011 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:27.539070 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:44:27.541249 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:27.541284 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:27.541900 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:27.542401 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:27.543812 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:27.545946 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:27.547306 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:27.548423 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:27.548632 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:27.552940 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:27.552952 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:27.552972 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:27.553032 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:27.561779 (+ 8747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:27.563037 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:27.563048 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:27.563058 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:27.563373 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:27.563941 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:27.563946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:27.563950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:27.564994 (+ 1044us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:27.565328 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:27.565943 (+ 615us) server_negotiation.cc:299] Negotiation successful
0810 11:44:27.566648 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":336,"threads_started":1}
I20260810 11:44:27.574164 25379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } }
I20260810 11:44:27.574698 25379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:27.574823 25379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3013c471fa8e479d916fc09c0fe871f5, State: Initialized, Role: FOLLOWER
I20260810 11:44:27.575381 25379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } }
I20260810 11:44:27.575591 25379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:27.575680 25379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:27.575807 25379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:27.578545 25379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } }
I20260810 11:44:27.579026 25379 leader_election.cc:302] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3013c471fa8e479d916fc09c0fe871f5; no voters:
I20260810 11:44:27.579620 25379 leader_election.cc:288] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:27.579792 25385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:27.580878 25385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3013c471fa8e479d916fc09c0fe871f5, State: Running, Role: LEADER
I20260810 11:44:27.581384 25385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } }
I20260810 11:44:27.581902 25379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:27.584507 25386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3013c471fa8e479d916fc09c0fe871f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } } }
I20260810 11:44:27.584534 25387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3013c471fa8e479d916fc09c0fe871f5. Latest consensus state: current_term: 1 leader_uuid: "3013c471fa8e479d916fc09c0fe871f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3013c471fa8e479d916fc09c0fe871f5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45699 } } }
I20260810 11:44:27.584929 25387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:27.584932 25386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:27.589090 25391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:27.595002 25391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:27.604717 25391 catalog_manager.cc:1347] Generated new cluster ID: 51c8469bf5b941499c4c1080105df38e
I20260810 11:44:27.604863 25391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:27.656417 25391 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:27.657892 25391 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:27.668915 25391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: Generated new TSK 0
I20260810 11:44:27.669960 25391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:27.734325 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:28.081005 25406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:28.081302 25406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:28.081465 25406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:28.089192 25406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:28.089300 25406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:28.089627 25406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:28.099766 25406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:28.101131 25406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:28.102555 25406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:28.115563 25412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:28.115643 25415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:28.115724 25411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:28.116371 25406 server_base.cc:1034] running on GCE node
I20260810 11:44:28.116966 25406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:28.117964 25406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:28.119169 25406 hybrid_clock.cc:648] HybridClock initialized: now 1786362268119120 us; error 38 us; skew 500 ppm
I20260810 11:44:28.119488 25406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:28 dist-test-slave-njxd krb5kdc[25291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:28.127398 25406 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:28.128795 25406 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:28.128948 25406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:28.129279 25406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:28.132359 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "7b002f83370948d19ec3196cfb966e80"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "f2a58378a4e65ab38188723072a1d63c"
server_key_iv: "284e00cd842f8e02189316a866944dc7"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.133194 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "7b002f83370948d19ec3196cfb966e80"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "f2a58378a4e65ab38188723072a1d63c"
server_key_iv: "284e00cd842f8e02189316a866944dc7"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.138616 25406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 11:44:28.142879 25421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.144357 25406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:28.144558 25406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "7b002f83370948d19ec3196cfb966e80"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "f2a58378a4e65ab38188723072a1d63c"
server_key_iv: "284e00cd842f8e02189316a866944dc7"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.144817 25406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:28.165541 25406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:28.170297 25406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:28.170667 25406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:28.171742 25406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:28.173121 25406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:28.173219 25406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.173302 25406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:28.173358 25406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.208014 25406 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:45385
I20260810 11:44:28.208072 25539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:45385 every 8 connection(s)
I20260810 11:44:28.208932 25406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:28.209543 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25406
I20260810 11:44:28.209846 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:28.210695 20854 external_mini_cluster.cc:1428] Setting key d88fa9528ecc7099aba2581a588bfc16
Aug 10 11:44:28 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:28.231181 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:28.211676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:37249 (local address 127.20.93.190:45699)
0810 11:44:28.212180 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:28.212187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:28.213255 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 11:44:28.215226 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:28.215234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:28.215325 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:28.215449 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:28.217091 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.218333 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.219982 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.220732 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.220885 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:28.225240 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:28.225245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:28.225248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:28.225281 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:28.227954 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:28.228736 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:28.228742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:28.228745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:28.228880 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:28.229314 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:28.229320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:28.229324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:28.229672 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:28.229827 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:28.230564 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 11:44:28.230801 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":246,"threads_started":1}
I20260810 11:44:28.232205 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:28.212270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:45699 (local address 127.20.93.129:37249)
0810 11:44:28.213072 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:28.213126 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:28.214927 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:28.215723 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:28.215736 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:28.216291 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:28.216897 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:28.216916 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.218504 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.218508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:28.219817 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:28.219831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.221032 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.221054 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:28.221388 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:28.222189 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:28.222222 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:28.224988 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:28.228118 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:28.228128 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:28.228146 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:28.228597 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:28.228997 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:28.229003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:28.229006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:28.229197 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:28.229802 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:28.229810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:28.230176 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 11:44:28.230454 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 11:44:28.231037 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":349,"threads_started":1}
I20260810 11:44:28.234057 25540 heartbeater.cc:347] Connected to a master server at 127.20.93.190:45699
I20260810 11:44:28.234423 25540 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:44:28.235219 25540 heartbeater.cc:511] Master 127.20.93.190:45699 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:44:28.237684 25342 ts_manager.cc:194] Registered new tserver with Master: 7b002f83370948d19ec3196cfb966e80 (127.20.93.129:45385)
I20260810 11:44:28.239238 25342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:37249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:28.266023 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:28.602451 25546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:28.602727 25546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:28.602882 25546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:28.610170 25546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:28.610296 25546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:28.610499 25546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:28.620517 25546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:28.621929 25546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:28.623346 25546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:28.636672 25553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:28.636883 25555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:28.637269 25552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:28.637586 25546 server_base.cc:1034] running on GCE node
I20260810 11:44:28.638185 25546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:28.639109 25546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:28.640350 25546 hybrid_clock.cc:648] HybridClock initialized: now 1786362268640305 us; error 40 us; skew 500 ppm
I20260810 11:44:28.640694 25546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:28 dist-test-slave-njxd krb5kdc[25291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:28.644155 25546 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:28.645529 25546 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:28.645709 25546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:28.646168 25546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:28.649530 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "91431a5b1034435691f78e7e0c25b240"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "0a6fa5874bbe86d646106db1e6fc5733"
server_key_iv: "9d62b59142c705dbbd4d0d33a712ed7f"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.650525 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "91431a5b1034435691f78e7e0c25b240"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "0a6fa5874bbe86d646106db1e6fc5733"
server_key_iv: "9d62b59142c705dbbd4d0d33a712ed7f"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.656162 25546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:28.660012 25562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.661409 25546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:28.661667 25546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "91431a5b1034435691f78e7e0c25b240"
format_stamp: "Formatted at 2026-08-10 11:44:28 on dist-test-slave-njxd"
server_key: "0a6fa5874bbe86d646106db1e6fc5733"
server_key_iv: "9d62b59142c705dbbd4d0d33a712ed7f"
server_key_version: "encryptionkey@0"
I20260810 11:44:28.661958 25546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:28.682482 25546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:28.687067 25546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:28.687366 25546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:28.688449 25546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:28.690135 25546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:28.690274 25546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.690405 25546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:28.690500 25546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:28.727950 25546 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:46343
I20260810 11:44:28.728077 25682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:46343 every 8 connection(s)
I20260810 11:44:28.729168 25546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:28.730206 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25546
I20260810 11:44:28.730417 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:28.731000 20854 external_mini_cluster.cc:1428] Setting key 20458fad6194acfc6c3a479bccd67d19
Aug 10 11:44:28 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:28.752875 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:28.732405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:53845 (local address 127.20.93.190:45699)
0810 11:44:28.732908 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:28.732915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:28.734306 (+ 1391us) server_negotiation.cc:407] Connection header received
0810 11:44:28.736273 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:28.736278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:28.736373 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:28.736503 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:28.738229 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.739023 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.740660 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.741117 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.741297 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:28.746183 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:28.746188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:28.746192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:28.746228 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:28.749088 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:28.750222 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:28.750226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:28.750228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:28.750368 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:28.750947 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:28.750950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:28.750952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:28.751221 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:28.751340 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:28.752385 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0810 11:44:28.752600 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":251,"threads_started":1}
I20260810 11:44:28.754654 25686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:28.733011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:45699 (local address 127.20.93.130:53845)
0810 11:44:28.734118 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:28.734169 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:28.735978 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:28.736774 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:28.736783 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:28.737352 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:28.738033 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:28.738052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.739202 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.739206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:28.740495 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:28.740508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:28.741388 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:28.741397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:28.741760 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:28.742615 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:28.742658 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:28.745896 (+ 3238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:28.749242 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:28.749256 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:28.749281 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:28.750090 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:28.750491 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:28.750499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:28.750505 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:28.750820 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:28.751373 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:28.751385 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:28.751891 (+ 506us) client_negotiation.cc:769] Sending connection context
0810 11:44:28.752244 (+ 353us) client_negotiation.cc:240] Negotiation successful
0810 11:44:28.752907 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":658,"threads_started":1}
I20260810 11:44:28.756349 25683 heartbeater.cc:347] Connected to a master server at 127.20.93.190:45699
I20260810 11:44:28.756681 25683 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
I20260810 11:44:28.757431 25683 heartbeater.cc:511] Master 127.20.93.190:45699 requested a full tablet report, sending...
I20260810 11:44:28.759070 25342 ts_manager.cc:194] Registered new tserver with Master: 91431a5b1034435691f78e7e0c25b240 (127.20.93.130:46343)
I20260810 11:44:28.759994 25342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:53845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:28.789357 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:29.144107 25689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:29.144464 25689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:29.144656 25689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:29.152289 25689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:29.152459 25689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:29.152649 25689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:29.162721 25689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:29.164152 25689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:29.165670 25689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:29.179557 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:29.179523 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:29.179616 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:29.180450 25689 server_base.cc:1034] running on GCE node
I20260810 11:44:29.181098 25689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:29.182368 25689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:29.183609 25689 hybrid_clock.cc:648] HybridClock initialized: now 1786362269183558 us; error 42 us; skew 500 ppm
I20260810 11:44:29.183959 25689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:29 dist-test-slave-njxd krb5kdc[25291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362269, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:29.187413 25689 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:29.188871 25689 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:29.189010 25689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:29.189368 25689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:29.192574 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1"
format_stamp: "Formatted at 2026-08-10 11:44:29 on dist-test-slave-njxd"
server_key: "bfc77b9f8aef8f56fc9174f10476cf68"
server_key_iv: "16fabf514fb403e7be21639eabac30d8"
server_key_version: "encryptionkey@0"
I20260810 11:44:29.193446 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1"
format_stamp: "Formatted at 2026-08-10 11:44:29 on dist-test-slave-njxd"
server_key: "bfc77b9f8aef8f56fc9174f10476cf68"
server_key_iv: "16fabf514fb403e7be21639eabac30d8"
server_key_version: "encryptionkey@0"
I20260810 11:44:29.199083 25689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:44:29.203491 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:29.205080 25689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:29.205292 25689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1"
format_stamp: "Formatted at 2026-08-10 11:44:29 on dist-test-slave-njxd"
server_key: "bfc77b9f8aef8f56fc9174f10476cf68"
server_key_iv: "16fabf514fb403e7be21639eabac30d8"
server_key_version: "encryptionkey@0"
I20260810 11:44:29.205549 25689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:29.242461 25540 heartbeater.cc:503] Master 127.20.93.190:45699 was elected leader, sending a full tablet report...
I20260810 11:44:29.256232 25689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:29.261155 25689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:29.261490 25689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:29.262640 25689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:29.264482 25689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:29.264633 25689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:29.264778 25689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:29.264873 25689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:29.301005 25689 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41343
I20260810 11:44:29.301061 25824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41343 every 8 connection(s)
I20260810 11:44:29.302003 25689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:29.305124 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25689
I20260810 11:44:29.305483 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:29.306037 20854 external_mini_cluster.cc:1428] Setting key 95ed51b5a0c5a57cd6bb5edb2e5ce542
Aug 10 11:44:29 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362269, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:29.324469 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.304730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:42605 (local address 127.20.93.190:45699)
0810 11:44:29.305299 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.305306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.306367 (+ 1061us) server_negotiation.cc:407] Connection header received
0810 11:44:29.308432 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.308436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.308508 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.308612 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:29.310397 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.311315 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.313250 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.313866 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.313974 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.317700 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:29.317705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:29.317709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:29.317758 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:29.320948 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.321954 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.321961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.321964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.322111 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.322572 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.322581 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.322583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.322880 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:29.323053 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.323835 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.324071 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":298,"threads_started":1}
I20260810 11:44:29.325497 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.305265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:45699 (local address 127.20.93.131:42605)
0810 11:44:29.306185 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.306231 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.308089 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.308897 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.308909 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.309505 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:29.310170 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.310189 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.311524 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.311528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.313098 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.313112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.314073 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.314078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.314296 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.314976 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:29.314999 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:29.317411 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:29.321098 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:29.321108 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:29.321128 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:29.321832 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:29.322231 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:29.322236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:29.322240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:29.322443 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:29.323026 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:29.323036 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:29.323439 (+ 403us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.323725 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.324322 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":405,"threads_started":1}
I20260810 11:44:29.326973 25825 heartbeater.cc:347] Connected to a master server at 127.20.93.190:45699
I20260810 11:44:29.327301 25825 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:29.328030 25825 heartbeater.cc:511] Master 127.20.93.190:45699 requested a full tablet report, sending...
I20260810 11:44:29.329564 25342 ts_manager.cc:194] Registered new tserver with Master: d7b80bb9c3ce4715b555e2ca360f5ce1 (127.20.93.131:41343)
I20260810 11:44:29.330478 25342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:42605
I20260810 11:44:29.334493 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:44:29.352660 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.339303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.20.93.190:45699)
0810 11:44:29.339490 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.339495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.340155 (+ 660us) server_negotiation.cc:407] Connection header received
0810 11:44:29.340317 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.340320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.340419 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.340511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:29.342205 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.342949 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.344451 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.344863 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.344965 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.347385 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:29.347390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:29.347393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:29.347429 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:29.350279 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.350943 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.350947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.350949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.351070 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.351423 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.351426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.351428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.351757 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:29.351935 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.352253 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.352430 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:44:29.356880 25342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:29.364632 25342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:29.392410 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.378116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45385 (local address 127.0.0.1:54398)
0810 11:44:29.380665 (+ 2549us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.380695 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.380845 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.384564 (+ 3719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.384571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.384605 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:29.384996 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.385010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.386817 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.386824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.389129 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.389164 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.391239 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.391246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.391461 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.391522 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.391695 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.391928 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2366,"mutex_wait_us":26,"thread_start_us":1042,"threads_started":1}
I20260810 11:44:29.392812 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.377916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41343 (local address 127.0.0.1:58312)
0810 11:44:29.379598 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.379661 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.379890 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.385043 (+ 5153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.385050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.385077 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:29.385731 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.385744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.387946 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.387954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.390178 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.390191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.391818 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.391825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.392009 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.392043 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.392135 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.392314 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1317,"mutex_wait_us":6,"spinlock_wait_cycles":11520,"thread_start_us":2342,"threads_started":1}
I20260810 11:44:29.393195 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.378883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54398 (local address 127.20.93.129:45385)
0810 11:44:29.383755 (+ 4872us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.383768 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.383806 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:44:29.383959 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.383971 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.384306 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.384539 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:29.385218 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.386672 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.389365 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.391009 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.391162 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.391437 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.392091 (+ 654us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.392272 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4660,"thread_start_us":4463,"threads_started":1}
I20260810 11:44:29.393231 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.381483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46343 (local address 127.0.0.1:55544)
0810 11:44:29.384093 (+ 2610us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.384127 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.384263 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.385212 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.385223 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.385261 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:29.385741 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.385750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.387729 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.387743 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.390206 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.390222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.392326 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.392332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.392538 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.392572 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.392655 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.392823 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2458,"mutex_wait_us":73,"spinlock_wait_cycles":1280,"thread_start_us":2313,"threads_started":1}
I20260810 11:44:29.393854 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.378299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.20.93.131:41343)
0810 11:44:29.384508 (+ 6209us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.384516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.384557 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:44:29.384645 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.384652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.384821 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.385957 (+ 1136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:29.386281 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.387805 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.390383 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.391588 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.391723 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.391942 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.393249 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.393443 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6028,"thread_start_us":5870,"threads_started":1}
I20260810 11:44:29.395124 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.382138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.20.93.130:46343)
0810 11:44:29.383145 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.383159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.384398 (+ 1239us) server_negotiation.cc:407] Connection header received
0810 11:44:29.384503 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.384511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.384757 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.384958 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:29.385938 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.387562 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.390422 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.391830 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.391967 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.392153 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.393622 (+ 1469us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.393810 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":586,"threads_started":1}
I20260810 11:44:29.397019 25472 tablet_service.cc:1467] Processing CreateTablet for tablet e4bd5da4587d4c2681afb332795c1884 (DEFAULT_TABLE table=test-table [id=a0f2701a202d44e58310b013d0643577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:29.397753 25756 tablet_service.cc:1467] Processing CreateTablet for tablet e4bd5da4587d4c2681afb332795c1884 (DEFAULT_TABLE table=test-table [id=a0f2701a202d44e58310b013d0643577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:29.398593 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4bd5da4587d4c2681afb332795c1884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:29.398620 25612 tablet_service.cc:1467] Processing CreateTablet for tablet e4bd5da4587d4c2681afb332795c1884 (DEFAULT_TABLE table=test-table [id=a0f2701a202d44e58310b013d0643577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:29.399200 25756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4bd5da4587d4c2681afb332795c1884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:29.399940 25612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4bd5da4587d4c2681afb332795c1884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:29.410593 25844 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Bootstrap starting.
I20260810 11:44:29.415027 25846 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Bootstrap starting.
I20260810 11:44:29.415093 25844 tablet_bootstrap.cc:654] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:29.416673 25844 log.cc:824] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:29.418311 25845 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Bootstrap starting.
I20260810 11:44:29.418957 25846 tablet_bootstrap.cc:654] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:29.420445 25846 log.cc:824] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:29.422224 25845 tablet_bootstrap.cc:654] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:29.423573 25845 log.cc:824] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:29.424280 25846 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: No bootstrap required, opened a new log
I20260810 11:44:29.424556 25846 ts_tablet_manager.cc:1397] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 11:44:29.425890 25844 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: No bootstrap required, opened a new log
I20260810 11:44:29.426205 25844 ts_tablet_manager.cc:1397] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20260810 11:44:29.426916 25845 tablet_bootstrap.cc:492] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: No bootstrap required, opened a new log
I20260810 11:44:29.427177 25845 ts_tablet_manager.cc:1397] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:44:29.435467 25844 raft_consensus.cc:348] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.435917 25844 raft_consensus.cc:374] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:29.435721 25846 raft_consensus.cc:348] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.436132 25844 raft_consensus.cc:729] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b002f83370948d19ec3196cfb966e80, State: Initialized, Role: FOLLOWER
I20260810 11:44:29.436201 25846 raft_consensus.cc:374] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:29.436326 25846 raft_consensus.cc:729] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b80bb9c3ce4715b555e2ca360f5ce1, State: Initialized, Role: FOLLOWER
I20260810 11:44:29.436950 25844 consensus_queue.cc:260] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.437031 25846 consensus_queue.cc:260] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.438020 25845 raft_consensus.cc:348] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.438465 25845 raft_consensus.cc:374] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:29.438570 25845 raft_consensus.cc:729] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91431a5b1034435691f78e7e0c25b240, State: Initialized, Role: FOLLOWER
I20260810 11:44:29.439167 25846 ts_tablet_manager.cc:1428] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260810 11:44:29.439280 25825 heartbeater.cc:503] Master 127.20.93.190:45699 was elected leader, sending a full tablet report...
I20260810 11:44:29.439354 25845 consensus_queue.cc:260] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.441099 25844 ts_tablet_manager.cc:1428] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 11:44:29.442304 25683 heartbeater.cc:503] Master 127.20.93.190:45699 was elected leader, sending a full tablet report...
I20260810 11:44:29.443027 25845 ts_tablet_manager.cc:1428] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
W20260810 11:44:29.461558 25541 tablet.cc:2369] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:44:29.481469 25684 tablet.cc:2369] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:44:29.553715 25826 tablet.cc:2369] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:29.629770 25850 raft_consensus.cc:482] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:29.630116 25850 raft_consensus.cc:504] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.631868 25850 leader_election.cc:288] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b002f83370948d19ec3196cfb966e80 (127.20.93.129:45385), 91431a5b1034435691f78e7e0c25b240 (127.20.93.130:46343)
I20260810 11:44:29.639479 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.632736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:45841 (local address 127.20.93.130:46343)
0810 11:44:29.632965 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.632973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.632995 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:44:29.633070 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.633075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.633161 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.633252 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:29.633781 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.634835 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.637244 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.638470 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.638651 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.638732 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.639076 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.639219 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:44:29.639490 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.632192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46343 (local address 127.20.93.131:45841)
0810 11:44:29.632460 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.632481 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.632623 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.633298 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.633305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.633338 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:29.633550 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.633559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.634962 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.634966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.637074 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.637089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.638660 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.638669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.638871 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.638912 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.639019 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.639167 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":1}
I20260810 11:44:29.639709 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.632896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:52637 (local address 127.20.93.129:45385)
0810 11:44:29.633109 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.633114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.633371 (+ 257us) server_negotiation.cc:407] Connection header received
0810 11:44:29.633498 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.633501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.633563 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.633712 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:29.634185 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.635168 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.637500 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.638729 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.638952 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.639025 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.639322 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.639456 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:44:29.639873 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.632319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45385 (local address 127.20.93.131:52637)
0810 11:44:29.633238 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:29.633257 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:29.633365 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:29.633731 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:29.633738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:29.633769 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:29.634028 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.634035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.635307 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.635312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.637331 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:29.637341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.638972 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.638979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:29.639154 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.639177 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:44:29.639292 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:44:29.639427 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":24,"thread_start_us":551,"threads_started":1}
I20260810 11:44:29.640987 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4bd5da4587d4c2681afb332795c1884" candidate_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b002f83370948d19ec3196cfb966e80" is_pre_election: true
I20260810 11:44:29.640980 25633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4bd5da4587d4c2681afb332795c1884" candidate_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91431a5b1034435691f78e7e0c25b240" is_pre_election: true
I20260810 11:44:29.641521 25633 raft_consensus.cc:2454] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b80bb9c3ce4715b555e2ca360f5ce1 in term 0.
I20260810 11:44:29.641521 25492 raft_consensus.cc:2454] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b80bb9c3ce4715b555e2ca360f5ce1 in term 0.
I20260810 11:44:29.642287 25707 leader_election.cc:302] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b002f83370948d19ec3196cfb966e80, d7b80bb9c3ce4715b555e2ca360f5ce1; no voters:
I20260810 11:44:29.642690 25850 raft_consensus.cc:2781] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:29.642836 25850 raft_consensus.cc:482] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:29.642937 25850 raft_consensus.cc:3037] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:29.645375 25850 raft_consensus.cc:504] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.646095 25850 leader_election.cc:288] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [CANDIDATE]: Term 1 election: Requested vote from peers 7b002f83370948d19ec3196cfb966e80 (127.20.93.129:45385), 91431a5b1034435691f78e7e0c25b240 (127.20.93.130:46343)
I20260810 11:44:29.646600 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4bd5da4587d4c2681afb332795c1884" candidate_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b002f83370948d19ec3196cfb966e80"
I20260810 11:44:29.646638 25633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4bd5da4587d4c2681afb332795c1884" candidate_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91431a5b1034435691f78e7e0c25b240"
I20260810 11:44:29.646818 25492 raft_consensus.cc:3037] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:29.646869 25633 raft_consensus.cc:3037] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:29.649286 25633 raft_consensus.cc:2454] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b80bb9c3ce4715b555e2ca360f5ce1 in term 1.
I20260810 11:44:29.649299 25492 raft_consensus.cc:2454] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b80bb9c3ce4715b555e2ca360f5ce1 in term 1.
I20260810 11:44:29.649787 25704 leader_election.cc:302] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91431a5b1034435691f78e7e0c25b240, d7b80bb9c3ce4715b555e2ca360f5ce1; no voters:
I20260810 11:44:29.650163 25850 raft_consensus.cc:2781] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:29.650663 25850 raft_consensus.cc:686] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 1 LEADER]: Becoming Leader. State: Replica: d7b80bb9c3ce4715b555e2ca360f5ce1, State: Running, Role: LEADER
I20260810 11:44:29.651206 25850 consensus_queue.cc:237] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } }
I20260810 11:44:29.656220 25341 catalog_manager.cc:5495] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7b80bb9c3ce4715b555e2ca360f5ce1 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b80bb9c3ce4715b555e2ca360f5ce1" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:29.690727 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.685222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.20.93.131:41343)
0810 11:44:29.685479 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.685485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.685516 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:44:29.685787 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.685791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.685874 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.686003 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:29.686344 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.687066 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.688474 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.688900 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.689042 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.689622 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:29.690126 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:29.690304 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.690374 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.690454 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:44:29.698925 25633 raft_consensus.cc:1261] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 1 FOLLOWER]: Refusing update from remote peer d7b80bb9c3ce4715b555e2ca360f5ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:29.698926 25492 raft_consensus.cc:1261] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 1 FOLLOWER]: Refusing update from remote peer d7b80bb9c3ce4715b555e2ca360f5ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:29.699947 25850 consensus_queue.cc:1035] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91431a5b1034435691f78e7e0c25b240" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:29.700273 25855 consensus_queue.cc:1035] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b002f83370948d19ec3196cfb966e80" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:29.713568 25860 mvcc.cc:205] Tried to move back new op lower bound from 7316939856676720640 to 7316939856495067136. Current Snapshot: MvccSnapshot[applied={T|T < 7316939856676720640}]
I20260810 11:44:29.735850 25342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:29.736186 25342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:29.740654 25342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3013c471fa8e479d916fc09c0fe871f5: Sending DeleteTablet for 3 replicas of tablet e4bd5da4587d4c2681afb332795c1884
I20260810 11:44:29.741930 25756 tablet_service.cc:1514] Processing DeleteTablet for tablet e4bd5da4587d4c2681afb332795c1884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:29 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:58312
I20260810 11:44:29.742128 25472 tablet_service.cc:1514] Processing DeleteTablet for tablet e4bd5da4587d4c2681afb332795c1884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:29 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:54398
I20260810 11:44:29.743129 25612 tablet_service.cc:1514] Processing DeleteTablet for tablet e4bd5da4587d4c2681afb332795c1884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:29 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:55544
I20260810 11:44:29.745466 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25406
I20260810 11:44:29.753940 25869 tablet_replica.cc:330] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: stopping tablet replica
I20260810 11:44:29.754456 25869 raft_consensus.cc:2229] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:29.755159 25868 tablet_replica.cc:330] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: stopping tablet replica
I20260810 11:44:29.756165 25869 raft_consensus.cc:2258] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:29.756846 25868 raft_consensus.cc:2229] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:29.758188 25870 tablet_replica.cc:330] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: stopping tablet replica
I20260810 11:44:29.758891 25870 raft_consensus.cc:2229] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:29.759253 25868 raft_consensus.cc:2258] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:29.759443 25870 raft_consensus.cc:2258] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:29.761870 25870 ts_tablet_manager.cc:1905] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:29.763459 25868 ts_tablet_manager.cc:1905] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:29.767293 25869 ts_tablet_manager.cc:1905] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:44:29 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:29.770386 25870 ts_tablet_manager.cc:1918] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:29.770627 25870 log.cc:1196] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/e4bd5da4587d4c2681afb332795c1884
I20260810 11:44:29.771816 25870 ts_tablet_manager.cc:1939] T e4bd5da4587d4c2681afb332795c1884 P 91431a5b1034435691f78e7e0c25b240: Deleting consensus metadata
I20260810 11:44:29.774745 25323 catalog_manager.cc:4841] TS 91431a5b1034435691f78e7e0c25b240 (127.20.93.130:46343): tablet e4bd5da4587d4c2681afb332795c1884 (table test-table [id=a0f2701a202d44e58310b013d0643577]) successfully deleted
I20260810 11:44:29.774915 25869 ts_tablet_manager.cc:1918] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:29.775089 25869 log.cc:1196] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/e4bd5da4587d4c2681afb332795c1884
I20260810 11:44:29.775648 25869 ts_tablet_manager.cc:1939] T e4bd5da4587d4c2681afb332795c1884 P 7b002f83370948d19ec3196cfb966e80: Deleting consensus metadata
I20260810 11:44:29.777529 25326 catalog_manager.cc:4841] TS 7b002f83370948d19ec3196cfb966e80 (127.20.93.129:45385): tablet e4bd5da4587d4c2681afb332795c1884 (table test-table [id=a0f2701a202d44e58310b013d0643577]) successfully deleted
I20260810 11:44:29.778426 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.746447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.20.93.129:45385)
0810 11:44:29.746645 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.746652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.755692 (+ 9040us) server_negotiation.cc:407] Connection header received
0810 11:44:29.757124 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.757131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.757218 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.757290 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:29.760518 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.761648 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.763692 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.764310 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.764414 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.770242 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:29.770258 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:29.770276 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:29.770316 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:29.774446 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.775691 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.775701 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.775768 (+ 67us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.775973 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.776484 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.776489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.776492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.776976 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:29.777277 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.777971 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.778163 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:44:29.779417 25868 ts_tablet_manager.cc:1918] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:29.779623 25868 log.cc:1196] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/e4bd5da4587d4c2681afb332795c1884
I20260810 11:44:29.779791 25535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:29.780215 25868 ts_tablet_manager.cc:1939] T e4bd5da4587d4c2681afb332795c1884 P d7b80bb9c3ce4715b555e2ca360f5ce1: Deleting consensus metadata
I20260810 11:44:29.782361 25327 catalog_manager.cc:4841] TS d7b80bb9c3ce4715b555e2ca360f5ce1 (127.20.93.131:41343): tablet e4bd5da4587d4c2681afb332795c1884 (table test-table [id=a0f2701a202d44e58310b013d0643577]) successfully deleted
I20260810 11:44:29.897667 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25406
I20260810 11:44:29.917733 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25546
Aug 10 11:44:29 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:29.935607 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:29.918640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.20.93.130:46343)
0810 11:44:29.918834 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:29.918838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:29.918857 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:44:29.918996 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:29.919000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:29.919079 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:29.919164 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:29.920861 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.921881 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.923720 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:29.924141 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:29.924244 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:29.929408 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:29.929424 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:29.929440 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:29.929482 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:29.932556 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.933331 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.933340 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.933342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.933474 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:29.933902 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:29.933908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:29.933911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:29.934335 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:29.934667 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:29.934987 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 11:44:29.935227 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:44:29.936424 25678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:30.046929 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25546
I20260810 11:44:30.073853 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25689
Aug 10 11:44:30 dist-test-slave-njxd krb5kdc[25291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:30.092532 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:30.074772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.20.93.131:41343)
0810 11:44:30.074977 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:30.074983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:30.075007 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:30.075139 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:30.075145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:30.075259 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:30.075385 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:30.076675 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:30.077937 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:30.079212 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:30.079926 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:30.080059 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:30.084961 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:30.084978 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:30.085005 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:30.085051 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:30.089450 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:30.090249 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:30.090260 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:30.090264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:30.090453 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:30.090830 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:30.090835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:30.090838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:30.091349 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:30.091688 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:30.091865 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 11:44:30.092095 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:44:30.093358 25820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:30.215235 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25689
I20260810 11:44:30.237473 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25308
I20260810 11:44:30.238801 25374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:30.349179 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25308
2026-08-10T11:44:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4689 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T11:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:30Z Disabled control of system clock
I20260810 11:44:30.428848 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44125
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 11:44:30.777050 25879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:30.777376 25879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:30.777566 25879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:30.785529 25879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:30.785701 25879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:30.785774 25879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:30.785880 25879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:30.785960 25879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:30.795523 25879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:30.796911 25879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:30.798830 25879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:30.811041 25885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:30.811002 25888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:30.811090 25886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:30.815101 25879 server_base.cc:1034] running on GCE node
I20260810 11:44:30.815918 25879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:30.817394 25879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:30.818625 25879 hybrid_clock.cc:648] HybridClock initialized: now 1786362270818591 us; error 59 us; skew 500 ppm
I20260810 11:44:30.819006 25879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:30.821333 25879 webserver.cc:460] Webserver started at http://127.20.93.190:36279/ using document root <none> and password file <none>
I20260810 11:44:30.822134 25879 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:30.822274 25879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:30.822610 25879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:30.825701 25879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "8a1be302b6ec4dba8303bab5b61da6bf"
format_stamp: "Formatted at 2026-08-10 11:44:30 on dist-test-slave-njxd"
server_key: "91c17d36dac1bcb9bbb968b00297a82f"
server_key_iv: "1afe1e0a5ec114989c7d70f881cd8f84"
server_key_version: "encryptionkey@0"
I20260810 11:44:30.826552 25879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "8a1be302b6ec4dba8303bab5b61da6bf"
format_stamp: "Formatted at 2026-08-10 11:44:30 on dist-test-slave-njxd"
server_key: "91c17d36dac1bcb9bbb968b00297a82f"
server_key_iv: "1afe1e0a5ec114989c7d70f881cd8f84"
server_key_version: "encryptionkey@0"
I20260810 11:44:30.831982 25879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:30.835475 25894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:30.836807 25879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:44:30.837002 25879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "8a1be302b6ec4dba8303bab5b61da6bf"
format_stamp: "Formatted at 2026-08-10 11:44:30 on dist-test-slave-njxd"
server_key: "91c17d36dac1bcb9bbb968b00297a82f"
server_key_iv: "1afe1e0a5ec114989c7d70f881cd8f84"
server_key_version: "encryptionkey@0"
I20260810 11:44:30.837240 25879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:30.843288 25879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:30.844074 25879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:30.844398 25879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:30.863093 25879 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44125
I20260810 11:44:30.863204 25949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44125 every 8 connection(s)
I20260810 11:44:30.864496 25879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:30.868372 25950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:30.873828 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25879
I20260810 11:44:30.874028 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:30.874897 20854 external_mini_cluster.cc:1428] Setting key bbeb571cf0eb96939193429a28bd8205
I20260810 11:44:30.880283 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: Bootstrap starting.
I20260810 11:44:30.883662 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:30.877212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.20.93.190:44125)
0810 11:44:30.878358 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:30.878370 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:30.878427 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:44:30.880060 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:30.880083 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:30.880550 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:30.880943 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:30.882048 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:30.882054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:30.882069 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:30.882083 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:30.882151 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:30.882184 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:30.882424 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:30.882569 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 11:44:30.882810 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":449,"threads_started":1}
I20260810 11:44:30.883713 25950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:30.885151 25950 log.cc:824] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:30.888092 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: No bootstrap required, opened a new log
I20260810 11:44:30.895431 25950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } }
I20260810 11:44:30.895820 25950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:30.895929 25950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1be302b6ec4dba8303bab5b61da6bf, State: Initialized, Role: FOLLOWER
I20260810 11:44:30.896504 25950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } }
I20260810 11:44:30.896775 25950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:30.896919 25950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:30.897102 25950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:30.899628 25950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } }
I20260810 11:44:30.900135 25950 leader_election.cc:302] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a1be302b6ec4dba8303bab5b61da6bf; no voters:
I20260810 11:44:30.900753 25950 leader_election.cc:288] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:30.900887 25956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:30.901654 25956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [term 1 LEADER]: Becoming Leader. State: Replica: 8a1be302b6ec4dba8303bab5b61da6bf, State: Running, Role: LEADER
I20260810 11:44:30.902174 25956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } }
I20260810 11:44:30.902976 25950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:30.905221 25958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1be302b6ec4dba8303bab5b61da6bf. Latest consensus state: current_term: 1 leader_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } } }
I20260810 11:44:30.905619 25958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:30.905196 25957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1be302b6ec4dba8303bab5b61da6bf" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44125 } } }
I20260810 11:44:30.906226 25957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:30.909116 25963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:30.914742 25963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:30.924612 25963 catalog_manager.cc:1347] Generated new cluster ID: 3fdfb56438b24cc1a824c54c40ee6500
I20260810 11:44:30.924767 25963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:30.940707 25963 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:30.941252 25963 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:30.950906 25963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: Generated new TSK 0
I20260810 11:44:30.952126 25963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:44:30.958091 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--builtin_ntp_servers=127.20.93.148:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 11:44:31.302889 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:31.303229 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:31.303416 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:31.311288 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:31.311429 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:31.311640 25976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:31.320860 25976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:31.322312 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:31.324131 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:31.337730 25982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:31.337776 25984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:31.338162 25981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:31.338611 25976 server_base.cc:1034] running on GCE node
I20260810 11:44:31.339198 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:31.340134 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:31.341361 25976 hybrid_clock.cc:648] HybridClock initialized: now 1786362271341321 us; error 56 us; skew 500 ppm
I20260810 11:44:31.341744 25976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:31.344182 25976 webserver.cc:460] Webserver started at http://127.20.93.129:44719/ using document root <none> and password file <none>
I20260810 11:44:31.344935 25976 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:31.345074 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:31.345419 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:31.348909 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "b36fa55bba04478b969d56cfbb37449e"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "358d20fedd38651e6b8f502c4858d3a5"
server_key_iv: "de7204e582378abc43941989109ad1d3"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.349817 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "b36fa55bba04478b969d56cfbb37449e"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "358d20fedd38651e6b8f502c4858d3a5"
server_key_iv: "de7204e582378abc43941989109ad1d3"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.355131 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 11:44:31.358701 25990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.360064 25976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:44:31.360246 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "b36fa55bba04478b969d56cfbb37449e"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "358d20fedd38651e6b8f502c4858d3a5"
server_key_iv: "de7204e582378abc43941989109ad1d3"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.360482 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:31.366528 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:31.367384 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:31.367673 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:31.368651 25976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:31.370182 25976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:31.370311 25976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.370452 25976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:31.370544 25976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.404634 25976 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:40901
I20260810 11:44:31.404655 26114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:40901 every 8 connection(s)
I20260810 11:44:31.406208 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:31.414526 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 25976
I20260810 11:44:31.414736 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:31.414721 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:31.409021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:34285 (local address 127.20.93.190:44125)
0810 11:44:31.409504 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:31.409510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:31.410822 (+ 1312us) server_negotiation.cc:407] Connection header received
0810 11:44:31.412478 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:31.412482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:31.412535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:31.412631 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:31.413950 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:31.413969 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:31.413972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:31.413987 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:31.414028 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:31.414040 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:31.414143 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:31.414392 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:44:31.414442 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":209,"threads_started":1}
I20260810 11:44:31.415602 20854 external_mini_cluster.cc:1428] Setting key 1fa70ad4f7124f3441a57a066272f98f
I20260810 11:44:31.416082 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:31.409748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44125 (local address 127.20.93.129:34285)
0810 11:44:31.410657 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:31.410700 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:31.412227 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:31.412803 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:31.412811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:31.412901 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:31.413664 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:31.413667 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:31.413706 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:31.413797 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:31.414152 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:31.414165 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:31.414173 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:44:31.414481 (+ 308us) client_negotiation.cc:240] Negotiation successful
0810 11:44:31.414795 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":249,"threads_started":1}
I20260810 11:44:31.417809 26115 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44125
I20260810 11:44:31.418125 26115 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:31.418718 26115 heartbeater.cc:511] Master 127.20.93.190:44125 requested a full tablet report, sending...
I20260810 11:44:31.418780 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--builtin_ntp_servers=127.20.93.148:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 11:44:31.420921 25911 ts_manager.cc:194] Registered new tserver with Master: b36fa55bba04478b969d56cfbb37449e (127.20.93.129:40901)
W20260810 11:44:31.760818 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:31.761127 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:31.761219 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:31.768942 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:31.769064 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:31.769179 26119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:31.778501 26119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:31.779841 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:31.781638 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:31.795506 26129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:31.795486 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:31.795974 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:31.796446 26119 server_base.cc:1034] running on GCE node
I20260810 11:44:31.797084 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:31.798193 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:31.799428 26119 hybrid_clock.cc:648] HybridClock initialized: now 1786362271799382 us; error 61 us; skew 500 ppm
I20260810 11:44:31.799764 26119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:31.802175 26119 webserver.cc:460] Webserver started at http://127.20.93.130:35079/ using document root <none> and password file <none>
I20260810 11:44:31.803014 26119 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:31.803153 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:31.803546 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:31.806847 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "b2e92c7516464de4924c7728f0d0d757"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "452a814c1cefb8a42ba123c7a9c74e06"
server_key_iv: "451e4b6cc9a3a2e87ea3710c236ec40d"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.807790 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "b2e92c7516464de4924c7728f0d0d757"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "452a814c1cefb8a42ba123c7a9c74e06"
server_key_iv: "451e4b6cc9a3a2e87ea3710c236ec40d"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.813267 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:44:31.816816 26135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.818256 26119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:31.818450 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "b2e92c7516464de4924c7728f0d0d757"
format_stamp: "Formatted at 2026-08-10 11:44:31 on dist-test-slave-njxd"
server_key: "452a814c1cefb8a42ba123c7a9c74e06"
server_key_iv: "451e4b6cc9a3a2e87ea3710c236ec40d"
server_key_version: "encryptionkey@0"
I20260810 11:44:31.818707 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:31.824518 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:31.825331 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:31.825673 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:31.826599 26119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:31.828035 26119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:31.828161 26119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.828301 26119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:31.828390 26119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:31.862323 26119 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:46439
I20260810 11:44:31.862463 26254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:46439 every 8 connection(s)
I20260810 11:44:31.863802 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:31.872824 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:31.866925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:45603 (local address 127.20.93.190:44125)
0810 11:44:31.867120 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:31.867124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:31.868739 (+ 1615us) server_negotiation.cc:407] Connection header received
0810 11:44:31.870517 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:31.870523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:31.870603 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:31.870742 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:31.872031 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:31.872038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:31.872043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:31.872062 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:31.872104 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:31.872115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:31.872267 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:31.872442 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 11:44:31.872510 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:44:31.873839 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:31.867566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44125 (local address 127.20.93.130:45603)
0810 11:44:31.868558 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:31.868615 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:31.870246 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:31.870966 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:31.870973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:31.871060 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:31.871746 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:31.871748 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:31.871785 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:31.871867 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:31.872215 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:31.872228 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:31.872237 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:44:31.872522 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 11:44:31.872773 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":333,"threads_started":1}
I20260810 11:44:31.874195 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26119
I20260810 11:44:31.874349 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:31.874910 20854 external_mini_cluster.cc:1428] Setting key 6f00ab6636c5928e018b09ed83ed642c
I20260810 11:44:31.877419 26255 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44125
I20260810 11:44:31.877784 26255 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:31.878186 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--builtin_ntp_servers=127.20.93.148:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 11:44:31.878471 26255 heartbeater.cc:511] Master 127.20.93.190:44125 requested a full tablet report, sending...
I20260810 11:44:31.879923 25910 ts_manager.cc:194] Registered new tserver with Master: b2e92c7516464de4924c7728f0d0d757 (127.20.93.130:46439)
W20260810 11:44:32.216408 26258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:32.216720 26258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:32.216890 26258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:32.225003 26258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:32.225163 26258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:32.225353 26258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:32.234854 26258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:32.236238 26258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:32.238157 26258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:32.251863 26267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:32.251914 26264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:32.251828 26263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:32.254954 26258 server_base.cc:1034] running on GCE node
I20260810 11:44:32.255609 26258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:32.256652 26258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:32.257895 26258 hybrid_clock.cc:648] HybridClock initialized: now 1786362272257857 us; error 60 us; skew 500 ppm
I20260810 11:44:32.258245 26258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:32.261437 26258 webserver.cc:460] Webserver started at http://127.20.93.131:41347/ using document root <none> and password file <none>
I20260810 11:44:32.262253 26258 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:32.262387 26258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:32.262760 26258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:32.265997 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "f4243a0a504e4c5ca2ca33e213e931a7"
format_stamp: "Formatted at 2026-08-10 11:44:32 on dist-test-slave-njxd"
server_key: "291f6733cbf1ec0e0a46792bd1807b75"
server_key_iv: "7b9deefadce92fc2aa56b678375d12d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:32.266848 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "f4243a0a504e4c5ca2ca33e213e931a7"
format_stamp: "Formatted at 2026-08-10 11:44:32 on dist-test-slave-njxd"
server_key: "291f6733cbf1ec0e0a46792bd1807b75"
server_key_iv: "7b9deefadce92fc2aa56b678375d12d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:32.272565 26258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:44:32.276436 26273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:32.277942 26258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:32.278138 26258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "f4243a0a504e4c5ca2ca33e213e931a7"
format_stamp: "Formatted at 2026-08-10 11:44:32 on dist-test-slave-njxd"
server_key: "291f6733cbf1ec0e0a46792bd1807b75"
server_key_iv: "7b9deefadce92fc2aa56b678375d12d7"
server_key_version: "encryptionkey@0"
I20260810 11:44:32.278386 26258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:32.284276 26258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:32.285138 26258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:32.285435 26258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:32.286474 26258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:32.287953 26258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:32.288077 26258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:32.288223 26258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:32.288312 26258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:32.322391 26258 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:39931
I20260810 11:44:32.322444 26388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:39931 every 8 connection(s)
I20260810 11:44:32.324047 26258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:32.332942 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.326929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:45099 (local address 127.20.93.190:44125)
0810 11:44:32.327115 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.327119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.328705 (+ 1586us) server_negotiation.cc:407] Connection header received
0810 11:44:32.330587 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.330593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.330676 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.330792 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.332142 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.332149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.332154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.332180 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.332216 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.332228 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.332360 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.332579 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.332656 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:44:32.333491 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26258
I20260810 11:44:32.333724 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:32.334208 20854 external_mini_cluster.cc:1428] Setting key 03354d19e1dbc624206c5301fbaa515f
I20260810 11:44:32.334537 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.327491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44125 (local address 127.20.93.131:45099)
0810 11:44:32.328444 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.328482 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.330297 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.331044 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.331053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.331154 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.331817 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.331820 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.331858 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.331948 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.332330 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.332342 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.332350 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.332670 (+ 320us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.332961 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":301,"threads_started":1}
I20260810 11:44:32.336230 26389 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44125
I20260810 11:44:32.336580 26389 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:32.337205 26389 heartbeater.cc:511] Master 127.20.93.190:44125 requested a full tablet report, sending...
I20260810 11:44:32.338658 25911 ts_manager.cc:194] Registered new tserver with Master: f4243a0a504e4c5ca2ca33e213e931a7 (127.20.93.131:39931)
I20260810 11:44:32.350225 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:44:32.357275 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.354736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.20.93.190:44125)
0810 11:44:32.354907 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.354912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.355518 (+ 606us) server_negotiation.cc:407] Connection header received
0810 11:44:32.355642 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.355646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.355702 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.355787 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.356749 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.356753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.356756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.356770 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.356801 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.356811 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.356908 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.356997 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.357050 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:44:32.361485 25911 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:32.368520 25911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:32.389678 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.382073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.20.93.130:46439)
0810 11:44:32.385666 (+ 3593us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.385680 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.386076 (+ 396us) server_negotiation.cc:407] Connection header received
0810 11:44:32.386174 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.386183 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.386476 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.386699 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.388594 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.388602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.388614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.388655 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.388739 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.388767 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.388925 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.389206 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.389309 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3311,"thread_start_us":3146,"threads_started":1}
I20260810 11:44:32.389750 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.382498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46439 (local address 127.0.0.1:59706)
0810 11:44:32.385765 (+ 3267us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.385790 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.385930 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.387036 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.387048 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.387161 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.388199 (+ 1038us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.388205 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.388260 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.388363 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.388979 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.388993 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.389004 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.389185 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.389277 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1936,"thread_start_us":1723,"threads_started":1}
I20260810 11:44:32.390738 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.381446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43034)
0810 11:44:32.383504 (+ 2058us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.383561 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.383763 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.387067 (+ 3304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.387073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.387151 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.388318 (+ 1167us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.388324 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.388345 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.388403 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.390065 (+ 1662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.390072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.390078 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.390192 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.390244 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"thread_start_us":1354,"threads_started":1}
I20260810 11:44:32.391206 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.384554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:39931 (local address 127.0.0.1:43286)
0810 11:44:32.387063 (+ 2509us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.387080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.387259 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.388145 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.388150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.388171 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.388881 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.388884 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.388898 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.388938 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.390549 (+ 1611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.390553 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.390558 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.390724 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.390772 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2366,"thread_start_us":2218,"threads_started":1}
I20260810 11:44:32.391227 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.385187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.20.93.131:39931)
0810 11:44:32.385969 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.385978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.387165 (+ 1187us) server_negotiation.cc:407] Connection header received
0810 11:44:32.387460 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.387471 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.387682 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.387815 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.389095 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.389102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.389114 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.390351 (+ 1237us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.390423 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.390449 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.390630 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.390702 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.390804 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":418,"threads_started":1}
I20260810 11:44:32.391795 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.381669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.20.93.129:40901)
0810 11:44:32.386446 (+ 4777us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.386455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.386488 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:44:32.386564 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.386574 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.386857 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.388869 (+ 2012us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.389816 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.389823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.389835 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.389861 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.389914 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.389933 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.391109 (+ 1176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.391188 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.391442 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4587,"thread_start_us":4468,"threads_started":1}
I20260810 11:44:32.394361 26184 tablet_service.cc:1467] Processing CreateTablet for tablet 72367c83f475425aab2292255e066763 (DEFAULT_TABLE table=test-table [id=bd12521a09aa493fa93723a00ce5d80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:32.395037 26043 tablet_service.cc:1467] Processing CreateTablet for tablet 72367c83f475425aab2292255e066763 (DEFAULT_TABLE table=test-table [id=bd12521a09aa493fa93723a00ce5d80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:32.394943 26322 tablet_service.cc:1467] Processing CreateTablet for tablet 72367c83f475425aab2292255e066763 (DEFAULT_TABLE table=test-table [id=bd12521a09aa493fa93723a00ce5d80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:32.395942 26184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72367c83f475425aab2292255e066763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:32.396270 26043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72367c83f475425aab2292255e066763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:32.396361 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72367c83f475425aab2292255e066763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:32.414695 26410 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Bootstrap starting.
I20260810 11:44:32.417142 26409 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Bootstrap starting.
I20260810 11:44:32.420359 26408 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: Bootstrap starting.
I20260810 11:44:32.420946 26409 tablet_bootstrap.cc:654] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:32.421960 26410 tablet_bootstrap.cc:654] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:32.422370 26409 log.cc:824] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:32.423467 26410 log.cc:824] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:32.424207 26115 heartbeater.cc:503] Master 127.20.93.190:44125 was elected leader, sending a full tablet report...
I20260810 11:44:32.424433 26408 tablet_bootstrap.cc:654] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:32.425987 26408 log.cc:824] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:32.428856 26410 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: No bootstrap required, opened a new log
I20260810 11:44:32.428856 26409 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: No bootstrap required, opened a new log
I20260810 11:44:32.429148 26409 ts_tablet_manager.cc:1397] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260810 11:44:32.429150 26410 ts_tablet_manager.cc:1397] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260810 11:44:32.431804 26408 tablet_bootstrap.cc:492] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: No bootstrap required, opened a new log
I20260810 11:44:32.432014 26408 ts_tablet_manager.cc:1397] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260810 11:44:32.439870 26408 raft_consensus.cc:348] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.440372 26408 raft_consensus.cc:374] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:32.440476 26408 raft_consensus.cc:729] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36fa55bba04478b969d56cfbb37449e, State: Initialized, Role: FOLLOWER
I20260810 11:44:32.440388 26409 raft_consensus.cc:348] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.440889 26409 raft_consensus.cc:374] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:32.441009 26409 raft_consensus.cc:729] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4243a0a504e4c5ca2ca33e213e931a7, State: Initialized, Role: FOLLOWER
I20260810 11:44:32.440896 26410 raft_consensus.cc:348] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.441413 26410 raft_consensus.cc:374] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:32.441507 26410 raft_consensus.cc:729] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e92c7516464de4924c7728f0d0d757, State: Initialized, Role: FOLLOWER
I20260810 11:44:32.441382 26408 consensus_queue.cc:260] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.441720 26409 consensus_queue.cc:260] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.442405 26410 consensus_queue.cc:260] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.443562 26408 ts_tablet_manager.cc:1428] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260810 11:44:32.444537 26389 heartbeater.cc:503] Master 127.20.93.190:44125 was elected leader, sending a full tablet report...
I20260810 11:44:32.445497 26409 ts_tablet_manager.cc:1428] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260810 11:44:32.449331 26255 heartbeater.cc:503] Master 127.20.93.190:44125 was elected leader, sending a full tablet report...
I20260810 11:44:32.449990 26410 ts_tablet_manager.cc:1428] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Time spent starting tablet: real 0.021s user 0.012s sys 0.008s
I20260810 11:44:32.486737 26415 raft_consensus.cc:482] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:32.487104 26415 raft_consensus.cc:504] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.488854 26415 leader_election.cc:288] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2e92c7516464de4924c7728f0d0d757 (127.20.93.130:46439), f4243a0a504e4c5ca2ca33e213e931a7 (127.20.93.131:39931)
I20260810 11:44:32.492842 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.489314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:51781 (local address 127.20.93.130:46439)
0810 11:44:32.489544 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.489551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.490616 (+ 1065us) server_negotiation.cc:407] Connection header received
0810 11:44:32.490741 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.490746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.490815 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.490927 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.492091 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.492097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.492102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.492123 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.492167 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.492186 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.492303 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.492449 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.492517 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:44:32.492930 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.489311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:57817 (local address 127.20.93.131:39931)
0810 11:44:32.489582 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.489608 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.490768 (+ 1160us) server_negotiation.cc:407] Connection header received
0810 11:44:32.490901 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.490907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.490968 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.491074 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.492271 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.492277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.492283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.492303 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.492347 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.492365 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.492491 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.492606 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.492666 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260810 11:44:32.493072 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.489629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46439 (local address 127.20.93.129:51781)
0810 11:44:32.490499 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.490515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.490622 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.490967 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.490971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.490997 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.491833 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.491838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.491859 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.491914 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.492300 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.492305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.492309 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.492473 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.492535 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":515,"threads_started":1}
I20260810 11:44:32.493218 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.489674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:39931 (local address 127.20.93.129:57817)
0810 11:44:32.490651 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.490669 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:32.490776 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:32.491125 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:32.491128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:32.491155 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:32.492030 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:44:32.492033 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:32.492050 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:44:32.492093 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:32.492476 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:32.492481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:32.492485 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 11:44:32.492642 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 11:44:32.492699 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"thread_start_us":391,"threads_started":1}
I20260810 11:44:32.494189 26206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72367c83f475425aab2292255e066763" candidate_uuid: "b36fa55bba04478b969d56cfbb37449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e92c7516464de4924c7728f0d0d757" is_pre_election: true
I20260810 11:44:32.494427 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72367c83f475425aab2292255e066763" candidate_uuid: "b36fa55bba04478b969d56cfbb37449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" is_pre_election: true
I20260810 11:44:32.494726 26206 raft_consensus.cc:2454] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36fa55bba04478b969d56cfbb37449e in term 0.
I20260810 11:44:32.494958 26343 raft_consensus.cc:2454] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36fa55bba04478b969d56cfbb37449e in term 0.
I20260810 11:44:32.495476 25994 leader_election.cc:302] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e92c7516464de4924c7728f0d0d757, b36fa55bba04478b969d56cfbb37449e; no voters:
I20260810 11:44:32.495900 26415 raft_consensus.cc:2781] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:32.496050 26415 raft_consensus.cc:482] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:32.496172 26415 raft_consensus.cc:3037] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:32.498477 26415 raft_consensus.cc:504] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.500342 26415 leader_election.cc:288] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [CANDIDATE]: Term 1 election: Requested vote from peers b2e92c7516464de4924c7728f0d0d757 (127.20.93.130:46439), f4243a0a504e4c5ca2ca33e213e931a7 (127.20.93.131:39931)
I20260810 11:44:32.500607 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72367c83f475425aab2292255e066763" candidate_uuid: "b36fa55bba04478b969d56cfbb37449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4243a0a504e4c5ca2ca33e213e931a7"
I20260810 11:44:32.500847 26343 raft_consensus.cc:3037] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:32.501140 26206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72367c83f475425aab2292255e066763" candidate_uuid: "b36fa55bba04478b969d56cfbb37449e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e92c7516464de4924c7728f0d0d757"
I20260810 11:44:32.501375 26206 raft_consensus.cc:3037] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:32.504390 26206 raft_consensus.cc:2454] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36fa55bba04478b969d56cfbb37449e in term 1.
I20260810 11:44:32.504998 26343 raft_consensus.cc:2454] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36fa55bba04478b969d56cfbb37449e in term 1.
I20260810 11:44:32.505565 25993 leader_election.cc:302] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36fa55bba04478b969d56cfbb37449e, f4243a0a504e4c5ca2ca33e213e931a7; no voters:
I20260810 11:44:32.506016 26415 raft_consensus.cc:2781] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:32.508658 26415 raft_consensus.cc:686] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 1 LEADER]: Becoming Leader. State: Replica: b36fa55bba04478b969d56cfbb37449e, State: Running, Role: LEADER
I20260810 11:44:32.509289 26415 consensus_queue.cc:237] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } }
I20260810 11:44:32.515128 25911 catalog_manager.cc:5495] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e reported cstate change: term changed from 0 to 1, leader changed from <none> to b36fa55bba04478b969d56cfbb37449e (127.20.93.129). New cstate: current_term: 1 leader_uuid: "b36fa55bba04478b969d56cfbb37449e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36fa55bba04478b969d56cfbb37449e" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:32.548954 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.546487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.20.93.129:40901)
0810 11:44:32.546655 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.546660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.546809 (+ 149us) server_negotiation.cc:407] Connection header received
0810 11:44:32.547065 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.547070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.547162 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.547276 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.548276 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.548280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.548283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.548299 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.548330 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.548343 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.548466 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.548594 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.548657 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 11:44:32.558480 26343 raft_consensus.cc:1261] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 1 FOLLOWER]: Refusing update from remote peer b36fa55bba04478b969d56cfbb37449e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:32.559206 26206 raft_consensus.cc:1261] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 1 FOLLOWER]: Refusing update from remote peer b36fa55bba04478b969d56cfbb37449e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:32.559719 26420 consensus_queue.cc:1035] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4243a0a504e4c5ca2ca33e213e931a7" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:32.560182 26415 consensus_queue.cc:1035] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e92c7516464de4924c7728f0d0d757" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:44:32.575778 26390 tablet.cc:2369] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:32.577075 26425 mvcc.cc:205] Tried to move back new op lower bound from 7316939868387205120 to 7316939868208082944. Current Snapshot: MvccSnapshot[applied={T|T < 7316939868387205120}]
I20260810 11:44:32.608454 25911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:32.608788 25911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:32.613183 25911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a1be302b6ec4dba8303bab5b61da6bf: Sending DeleteTablet for 3 replicas of tablet 72367c83f475425aab2292255e066763
I20260810 11:44:32.614528 26184 tablet_service.cc:1514] Processing DeleteTablet for tablet 72367c83f475425aab2292255e066763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:32 UTC) from {username='slave'} at 127.0.0.1:59706
I20260810 11:44:32.614904 26043 tablet_service.cc:1514] Processing DeleteTablet for tablet 72367c83f475425aab2292255e066763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:32 UTC) from {username='slave'} at 127.0.0.1:43034
I20260810 11:44:32.615639 26322 tablet_service.cc:1514] Processing DeleteTablet for tablet 72367c83f475425aab2292255e066763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:32 UTC) from {username='slave'} at 127.0.0.1:43286
I20260810 11:44:32.620360 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25976
I20260810 11:44:32.639768 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.621930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.20.93.129:40901)
0810 11:44:32.622168 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.622176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.626050 (+ 3874us) server_negotiation.cc:407] Connection header received
0810 11:44:32.626293 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.626300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.626380 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.626485 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.638948 (+ 12463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.638957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.638964 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.638998 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.639050 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.639070 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.639211 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.639314 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.639385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:44:32.641240 26110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:32.649894 26437 tablet_replica.cc:330] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: stopping tablet replica
I20260810 11:44:32.650591 26437 raft_consensus.cc:2229] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:32.651298 26437 raft_consensus.cc:2258] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:32.654027 26435 tablet_replica.cc:330] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: stopping tablet replica
I20260810 11:44:32.654568 26435 raft_consensus.cc:2229] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:32.655158 26435 raft_consensus.cc:2258] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:32.655781 26437 ts_tablet_manager.cc:1905] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:32.657243 26435 ts_tablet_manager.cc:1905] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:32.660753 26437 ts_tablet_manager.cc:1918] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:32.660938 26437 log.cc:1196] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/72367c83f475425aab2292255e066763
I20260810 11:44:32.661408 26437 ts_tablet_manager.cc:1939] T 72367c83f475425aab2292255e066763 P f4243a0a504e4c5ca2ca33e213e931a7: Deleting consensus metadata
I20260810 11:44:32.663172 25897 catalog_manager.cc:4841] TS f4243a0a504e4c5ca2ca33e213e931a7 (127.20.93.131:39931): tablet 72367c83f475425aab2292255e066763 (table test-table [id=bd12521a09aa493fa93723a00ce5d80a]) successfully deleted
I20260810 11:44:32.664775 26435 ts_tablet_manager.cc:1918] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:32.664973 26435 log.cc:1196] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/72367c83f475425aab2292255e066763
I20260810 11:44:32.665555 26435 ts_tablet_manager.cc:1939] T 72367c83f475425aab2292255e066763 P b2e92c7516464de4924c7728f0d0d757: Deleting consensus metadata
I20260810 11:44:32.667654 25898 catalog_manager.cc:4841] TS b2e92c7516464de4924c7728f0d0d757 (127.20.93.130:46439): tablet 72367c83f475425aab2292255e066763 (table test-table [id=bd12521a09aa493fa93723a00ce5d80a]) successfully deleted
I20260810 11:44:32.750437 26436 tablet_replica.cc:330] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e: stopping tablet replica
I20260810 11:44:32.750962 26436 raft_consensus.cc:2229] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:32.751586 26436 raft_consensus.cc:2258] T 72367c83f475425aab2292255e066763 P b36fa55bba04478b969d56cfbb37449e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:32.752350 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25976
W20260810 11:44:32.772291 25897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.93.129:40901 (error 108)
I20260810 11:44:32.772444 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26119
I20260810 11:44:32.774364 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.773808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43064)
0810 11:44:32.774009 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.774158 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:44:32.774659 25897 catalog_manager.cc:4584] TS b36fa55bba04478b969d56cfbb37449e (127.20.93.129:40901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72367c83f475425aab2292255e066763: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
I20260810 11:44:32.775696 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.773300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.20.93.130:46439)
0810 11:44:32.773494 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.773498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.773551 (+ 53us) server_negotiation.cc:407] Connection header received
0810 11:44:32.773787 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.773793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.773918 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.774033 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.775104 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.775108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.775111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.775126 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.775160 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.775172 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.775278 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.775348 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.775394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:44:32.776386 26250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:32.828339 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.827764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43070)
0810 11:44:32.828013 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.828130 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 11:44:32.829394 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.829008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43080)
0810 11:44:32.829158 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.829256 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 11:44:32.865274 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26119
I20260810 11:44:32.890242 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.889433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43082)
0810 11:44:32.889905 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.890025 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
I20260810 11:44:32.890223 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26258
I20260810 11:44:32.891455 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.891011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43090)
0810 11:44:32.891209 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.891317 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:44:32.893301 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.891017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.20.93.131:39931)
0810 11:44:32.891298 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:32.891305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:32.891327 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:44:32.891505 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:32.891512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:32.891639 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:32.891741 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:32.892649 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:32.892659 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:32.892662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:32.892678 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:32.892709 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:32.892721 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:32.892870 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:32.892967 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 11:44:32.893028 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260810 11:44:32.893994 26384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:32.972220 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.971736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43092)
0810 11:44:32.971964 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.972061 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 11:44:32.973407 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:32.972988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40901 (local address 127.0.0.1:43106)
0810 11:44:32.973197 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:32.973290 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 11:44:32.988018 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26258
I20260810 11:44:33.008183 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 25879
I20260810 11:44:33.009532 25943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:33.120702 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 25879
2026-08-10T11:44:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2770 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 11:44:33.165740 20854 mini_oidc.cc:152] Starting JWKS server
I20260810 11:44:33.168648 20854 webserver.cc:460] Webserver started at https://127.0.0.1:37795/ using document root <none> and password file <none>
I20260810 11:44:33.169004 20854 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:44:33.169703 20854 webserver.cc:460] Webserver started at http://127.0.0.1:40115/ using document root <none> and password file <none>
2026-08-10T11:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:33Z Disabled control of system clock
I20260810 11:44:33.203898 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33537
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:40403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:33537
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37795/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/testchainca.cert with env {}
W20260810 11:44:33.540014 26449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:33.540315 26449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:33.540486 26449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:33.547911 26449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:33.548017 26449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:33.548091 26449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:33.548193 26449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:33.548275 26449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:44:33.548367 26449 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:44:33.548442 26449 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37795/jwks.json
I20260810 11:44:33.558735 26449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:40403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:33537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37795/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:33.560074 26449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:33.561458 26449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:33.574000 26458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:33.574048 26455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:33.574087 26456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:33.574929 26449 server_base.cc:1034] running on GCE node
I20260810 11:44:33.575634 26449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:33.577003 26449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:33.578235 26449 hybrid_clock.cc:648] HybridClock initialized: now 1786362273578204 us; error 50 us; skew 500 ppm
I20260810 11:44:33.578598 26449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:33.585755 26449 webserver.cc:460] Webserver started at http://127.20.93.190:40793/ using document root <none> and password file <none>
I20260810 11:44:33.586489 26449 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:33.586642 26449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:33.586975 26449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:33.590020 26449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "8549cfa8b8fa46c7bacc3489f7a33dca"
format_stamp: "Formatted at 2026-08-10 11:44:33 on dist-test-slave-njxd"
server_key: "e23e1935af80a95570068fe3d81dc44a"
server_key_iv: "63c239c9b86b43b6526477cf59b4aaee"
server_key_version: "encryptionkey@0"
I20260810 11:44:33.590842 26449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "8549cfa8b8fa46c7bacc3489f7a33dca"
format_stamp: "Formatted at 2026-08-10 11:44:33 on dist-test-slave-njxd"
server_key: "e23e1935af80a95570068fe3d81dc44a"
server_key_iv: "63c239c9b86b43b6526477cf59b4aaee"
server_key_version: "encryptionkey@0"
I20260810 11:44:33.595966 26449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 11:44:33.599406 26464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:33.600801 26449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:33.600996 26449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "8549cfa8b8fa46c7bacc3489f7a33dca"
format_stamp: "Formatted at 2026-08-10 11:44:33 on dist-test-slave-njxd"
server_key: "e23e1935af80a95570068fe3d81dc44a"
server_key_iv: "63c239c9b86b43b6526477cf59b4aaee"
server_key_version: "encryptionkey@0"
I20260810 11:44:33.601217 26449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:33.620355 26449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:33.621223 26449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:33.621548 26449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:33.640167 26449 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:33537
I20260810 11:44:33.640187 26521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:33537 every 8 connection(s)
I20260810 11:44:33.641603 26449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:33.644964 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26449
I20260810 11:44:33.645212 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:33.645788 26522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:33.645965 20854 external_mini_cluster.cc:1428] Setting key c814331f85aa837f5a2ca5c9f237ee60
I20260810 11:44:33.659948 26522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca: Bootstrap starting.
I20260810 11:44:33.660302 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:33.647967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37038 (local address 127.20.93.190:33537)
0810 11:44:33.649050 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:33.649061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:33.649107 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:44:33.650715 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:33.650739 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:33.651210 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:33.651583 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:33.652258 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.653774 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.655760 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.656447 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.656575 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:33.657928 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:33.657935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:33.657951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:33.658509 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:33.658584 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:33.658609 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:33.658694 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:33.658951 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 11:44:33.659308 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":397,"threads_started":1}
I20260810 11:44:33.663497 26522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:33.664225 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:44:33.664714 26522 log.cc:824] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:33.667845 26522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca: No bootstrap required, opened a new log
I20260810 11:44:33.675671 26522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } }
I20260810 11:44:33.676047 26522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:33.676127 26522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8549cfa8b8fa46c7bacc3489f7a33dca, State: Initialized, Role: FOLLOWER
I20260810 11:44:33.676654 26522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } }
I20260810 11:44:33.676846 26522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:33.676946 26522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:33.677067 26522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:33.679488 26522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } }
I20260810 11:44:33.680094 26522 leader_election.cc:302] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8549cfa8b8fa46c7bacc3489f7a33dca; no voters:
I20260810 11:44:33.680717 26522 leader_election.cc:288] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:33.680881 26527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:33.681629 26527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [term 1 LEADER]: Becoming Leader. State: Replica: 8549cfa8b8fa46c7bacc3489f7a33dca, State: Running, Role: LEADER
I20260810 11:44:33.682228 26527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } }
I20260810 11:44:33.682898 26522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:33.685469 26528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } } }
I20260810 11:44:33.685714 26530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8549cfa8b8fa46c7bacc3489f7a33dca. Latest consensus state: current_term: 1 leader_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549cfa8b8fa46c7bacc3489f7a33dca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33537 } } }
I20260810 11:44:33.685940 26528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:33.685982 26530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:33.688387 26536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:33.693890 26536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:33.703611 26536 catalog_manager.cc:1347] Generated new cluster ID: 4e8728c841e24c1fb32e7255aa4b4e80
I20260810 11:44:33.703756 26536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:33.714186 26536 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:33.715962 26536 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:33.731182 26536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8549cfa8b8fa46c7bacc3489f7a33dca: Generated new TSK 0
I20260810 11:44:33.732746 26536 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 11:44:33.778760 26553 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 11:44:33.778940 26553 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:44:33.781507 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:33.757413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.20.93.190:33537)
0810 11:44:33.757659 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:33.757666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:33.758133 (+ 467us) server_negotiation.cc:407] Connection header received
0810 11:44:33.758485 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:33.758491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:33.758590 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:33.758757 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:44:33.759243 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.760394 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.761874 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.762594 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.762721 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:33.763256 (+ 535us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:44:33.780788 (+ 17532us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 11:44:33.781175 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 11:44:33.781674 26525 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 11:44:33.783175 26552 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:33.757264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33537 (local address 127.0.0.1:37048)
0810 11:44:33.758029 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:33.758047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:33.758216 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:33.758780 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:33.758783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:33.758793 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:44:33.759106 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:33.759113 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.760533 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.760537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:33.761735 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:33.761747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:33.762776 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:33.762781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:33.762899 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:33.762958 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:44:33.781084 (+ 18126us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 11:44:33.781342 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.93.190:33537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":428,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.93.190:33537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 11:44:33.785152 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26449
I20260810 11:44:33.786357 26517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:33.881328 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26449
2026-08-10T11:44:33Z chronyd exiting
I20260810 11:44:33.935487 20854 test_util.cc:175] -----------------------------------------------
I20260810 11:44:33.935635 20854 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786362228785499-20854-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (774 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 11:44:33.940407 20854 mini_oidc.cc:152] Starting JWKS server
I20260810 11:44:33.942368 20854 webserver.cc:460] Webserver started at https://127.0.0.1:46271/ using document root <none> and password file <none>
I20260810 11:44:33.942619 20854 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:44:33.943291 20854 webserver.cc:460] Webserver started at http://127.0.0.1:37433/ using document root <none> and password file <none>
2026-08-10T11:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:33Z Disabled control of system clock
I20260810 11:44:33.978760 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38881
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38881
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46271/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/testchainca.cert with env {}
W20260810 11:44:34.317057 26561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:34.317428 26561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:34.317652 26561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:34.325139 26561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:34.325302 26561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:34.325415 26561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:34.325526 26561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:34.325639 26561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:44:34.325729 26561 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:44:34.325809 26561 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46271/jwks.json
I20260810 11:44:34.336248 26561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46271/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:34.337813 26561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:34.339219 26561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:34.351734 26568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:34.351879 26571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:34.351900 26569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:34.352797 26561 server_base.cc:1034] running on GCE node
I20260810 11:44:34.353658 26561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:34.355286 26561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:34.356535 26561 hybrid_clock.cc:648] HybridClock initialized: now 1786362274356496 us; error 55 us; skew 500 ppm
I20260810 11:44:34.356920 26561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:34.359350 26561 webserver.cc:460] Webserver started at http://127.20.93.190:33311/ using document root <none> and password file <none>
I20260810 11:44:34.360095 26561 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:34.360240 26561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:34.360590 26561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:34.363662 26561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "108c7fcc64244a84a015ad6a5913de75"
format_stamp: "Formatted at 2026-08-10 11:44:34 on dist-test-slave-njxd"
server_key: "b4ebc7b16a6837b4f1057f8027c74ecb"
server_key_iv: "8f218902118b4c62f6fe017f417536bf"
server_key_version: "encryptionkey@0"
I20260810 11:44:34.364522 26561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "108c7fcc64244a84a015ad6a5913de75"
format_stamp: "Formatted at 2026-08-10 11:44:34 on dist-test-slave-njxd"
server_key: "b4ebc7b16a6837b4f1057f8027c74ecb"
server_key_iv: "8f218902118b4c62f6fe017f417536bf"
server_key_version: "encryptionkey@0"
I20260810 11:44:34.369933 26561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 11:44:34.373530 26577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:34.375102 26561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:34.375324 26561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "108c7fcc64244a84a015ad6a5913de75"
format_stamp: "Formatted at 2026-08-10 11:44:34 on dist-test-slave-njxd"
server_key: "b4ebc7b16a6837b4f1057f8027c74ecb"
server_key_iv: "8f218902118b4c62f6fe017f417536bf"
server_key_version: "encryptionkey@0"
I20260810 11:44:34.375568 26561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:34.394563 26561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:34.395529 26561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:34.395881 26561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:34.415421 26561 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38881
I20260810 11:44:34.415467 26630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38881 every 8 connection(s)
I20260810 11:44:34.416919 26561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:34.421029 26631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:34.421962 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26561
I20260810 11:44:34.422173 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:34.423062 20854 external_mini_cluster.cc:1428] Setting key 9ec1ed9b40421d9edb2f55aa0ded64e1
I20260810 11:44:34.434918 26631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75: Bootstrap starting.
I20260810 11:44:34.437201 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:34.425267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.20.93.190:38881)
0810 11:44:34.426387 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:34.426403 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:34.426451 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:44:34.428141 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:34.428165 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:34.428698 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:34.429146 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:34.429788 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.431083 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.432458 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.433216 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.433440 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:34.434741 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:34.434749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:34.434764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:34.435375 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:34.435463 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:34.435489 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:34.435620 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:34.435865 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 11:44:34.436199 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":415,"threads_started":1}
I20260810 11:44:34.438484 26631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:34.439630 26631 log.cc:824] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:34.441334 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:44:34.442853 26631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75: No bootstrap required, opened a new log
I20260810 11:44:34.450361 26631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } }
I20260810 11:44:34.450832 26631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:34.450958 26631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108c7fcc64244a84a015ad6a5913de75, State: Initialized, Role: FOLLOWER
I20260810 11:44:34.451565 26631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } }
I20260810 11:44:34.451785 26631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:34.451889 26631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:34.452011 26631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:34.454282 26631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } }
I20260810 11:44:34.454896 26631 leader_election.cc:302] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 108c7fcc64244a84a015ad6a5913de75; no voters:
I20260810 11:44:34.455430 26631 leader_election.cc:288] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:34.455608 26637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:34.456358 26637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [term 1 LEADER]: Becoming Leader. State: Replica: 108c7fcc64244a84a015ad6a5913de75, State: Running, Role: LEADER
I20260810 11:44:34.456918 26637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } }
I20260810 11:44:34.458019 26631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:34.460105 26640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 108c7fcc64244a84a015ad6a5913de75. Latest consensus state: current_term: 1 leader_uuid: "108c7fcc64244a84a015ad6a5913de75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } } }
I20260810 11:44:34.460568 26640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:34.461241 26639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "108c7fcc64244a84a015ad6a5913de75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108c7fcc64244a84a015ad6a5913de75" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38881 } } }
I20260810 11:44:34.461635 26639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:34.463740 26648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:34.468700 26648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:34.477460 26648 catalog_manager.cc:1347] Generated new cluster ID: a003d970f1344c5f86c2badc82909b44
I20260810 11:44:34.477624 26648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:34.499526 26648 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:34.500967 26648 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:34.513800 26648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 108c7fcc64244a84a015ad6a5913de75: Generated new TSK 0
I20260810 11:44:34.514777 26648 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 11:44:34.559561 26666 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 11:44:34.559711 26666 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:44:34.560935 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:34.533564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.20.93.190:38881)
0810 11:44:34.533821 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:34.533830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:34.534225 (+ 395us) server_negotiation.cc:407] Connection header received
0810 11:44:34.534490 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:34.534494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:34.534594 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:34.534766 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:44:34.535282 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.536011 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.537427 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.537975 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.538124 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:34.538636 (+ 512us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:44:34.560163 (+ 21527us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 11:44:34.560609 (+ 446us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 11:44:34.561084 26635 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 11:44:34.561183 26665 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:34.533446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38881 (local address 127.0.0.1:50510)
0810 11:44:34.534113 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:34.534129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:34.534304 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:34.534787 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:34.534793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:34.534807 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:44:34.535089 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:34.535099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.536165 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.536178 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:34.537260 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:34.537273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:34.538152 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:34.538159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:34.538328 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:34.538364 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:44:34.560455 (+ 22091us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 11:44:34.560762 (+ 307us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.93.190:38881: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":401,"threads_started":1}
I20260810 11:44:34.562988 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26561
I20260810 11:44:34.564271 26626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:34.661548 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26561
2026-08-10T11:44:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (782 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 11:44:34.722010 20854 mini_oidc.cc:152] Starting JWKS server
I20260810 11:44:34.723954 20854 webserver.cc:460] Webserver started at https://127.0.0.1:43791/ using document root <none> and password file <none>
I20260810 11:44:34.724217 20854 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:44:34.724876 20854 webserver.cc:460] Webserver started at http://127.0.0.1:46547/ using document root <none> and password file <none>
2026-08-10T11:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:34Z Disabled control of system clock
I20260810 11:44:34.761365 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38049
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:35181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38049
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 11:44:35.117108 26673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:35.117439 26673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:35.117627 26673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:35.125285 26673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:35.125411 26673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:35.125478 26673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:35.125608 26673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:35.125691 26673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:44:35.125809 26673 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:44:35.125887 26673 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43791/jwks.json
I20260810 11:44:35.135607 26673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:35.137042 26673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:35.138998 26673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:35.151262 26678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:35.151217 26681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:35.151320 26679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:35.152261 26673 server_base.cc:1034] running on GCE node
I20260810 11:44:35.153178 26673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:35.154875 26673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:35.156163 26673 hybrid_clock.cc:648] HybridClock initialized: now 1786362275156112 us; error 66 us; skew 500 ppm
I20260810 11:44:35.156526 26673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:35.158970 26673 webserver.cc:460] Webserver started at http://127.20.93.190:36889/ using document root <none> and password file <none>
I20260810 11:44:35.159688 26673 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:35.159795 26673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:35.160086 26673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:35.163297 26673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "ef89dbc0a46d462bb5cf6cb9b555df71"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "401f967c2a5f16cd780e66feb75e7cad"
server_key_iv: "9f22a212aaaf12a219b8c3c1daa99467"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.164131 26673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "ef89dbc0a46d462bb5cf6cb9b555df71"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "401f967c2a5f16cd780e66feb75e7cad"
server_key_iv: "9f22a212aaaf12a219b8c3c1daa99467"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.170107 26673 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:44:35.173925 26687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:35.175514 26673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:35.175715 26673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "ef89dbc0a46d462bb5cf6cb9b555df71"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "401f967c2a5f16cd780e66feb75e7cad"
server_key_iv: "9f22a212aaaf12a219b8c3c1daa99467"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.175976 26673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:35.193544 26673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:35.194636 26673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:35.195009 26673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:35.214901 26673 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38049
I20260810 11:44:35.214921 26739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38049 every 8 connection(s)
I20260810 11:44:35.216503 26673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:35.220976 26740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:35.224850 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26673
I20260810 11:44:35.225093 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:35.226009 20854 external_mini_cluster.cc:1428] Setting key 6a35bc5600753ce752244cd49d745687
I20260810 11:44:35.234735 26740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71: Bootstrap starting.
I20260810 11:44:35.238303 26740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:35.239579 26740 log.cc:824] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:35.240509 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:35.228126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.20.93.190:38049)
0810 11:44:35.229182 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:35.229193 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:35.229238 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:44:35.231012 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:35.231039 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:35.231584 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:35.232045 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:35.232691 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.234087 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.235582 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.236308 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.236539 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:35.237857 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:35.237870 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:35.237886 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:35.238499 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:35.238613 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:35.238645 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:35.238812 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:35.239048 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 11:44:35.239391 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":372,"threads_started":1}
I20260810 11:44:35.242741 26740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71: No bootstrap required, opened a new log
I20260810 11:44:35.244544 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:44:35.251552 26740 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } }
I20260810 11:44:35.252046 26740 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:35.252169 26740 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef89dbc0a46d462bb5cf6cb9b555df71, State: Initialized, Role: FOLLOWER
I20260810 11:44:35.252784 26740 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } }
I20260810 11:44:35.253075 26740 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:35.253237 26740 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:35.253449 26740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:35.255968 26740 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } }
I20260810 11:44:35.256606 26740 leader_election.cc:302] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef89dbc0a46d462bb5cf6cb9b555df71; no voters:
I20260810 11:44:35.257241 26740 leader_election.cc:288] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:35.257431 26745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:35.258209 26745 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [term 1 LEADER]: Becoming Leader. State: Replica: ef89dbc0a46d462bb5cf6cb9b555df71, State: Running, Role: LEADER
I20260810 11:44:35.258805 26745 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } }
I20260810 11:44:35.259577 26740 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:35.261673 26746 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } } }
I20260810 11:44:35.262096 26746 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:35.261914 26747 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef89dbc0a46d462bb5cf6cb9b555df71. Latest consensus state: current_term: 1 leader_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef89dbc0a46d462bb5cf6cb9b555df71" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38049 } } }
I20260810 11:44:35.262343 26747 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:35.264387 26751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:35.269167 26751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:35.279541 26751 catalog_manager.cc:1347] Generated new cluster ID: 00432e0574514a43b3eeb14130b90bce
I20260810 11:44:35.279687 26751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:35.299719 26751 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:35.301829 26751 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:35.318819 26751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef89dbc0a46d462bb5cf6cb9b555df71: Generated new TSK 0
I20260810 11:44:35.319788 26751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:44:35.342847 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:35.334827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.20.93.190:38049)
0810 11:44:35.335077 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:35.335087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:35.335719 (+ 632us) server_negotiation.cc:407] Connection header received
0810 11:44:35.335993 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:35.335998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:35.336074 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:35.336239 (+ 165us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:44:35.336756 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.337494 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.338918 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.339422 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.339569 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:35.340133 (+ 564us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:44:35.342154 (+ 2021us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 11:44:35.342517 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 11:44:35.343077 26774 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:35.334672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38049 (local address 127.0.0.1:59480)
0810 11:44:35.335601 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:35.335623 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:35.335804 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:35.336267 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:35.336273 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:35.336286 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:44:35.336579 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:35.336588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.337669 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.337676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:35.338769 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:35.338780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:35.339633 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:35.339642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:35.339805 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:35.339847 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:44:35.342365 (+ 2518us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 11:44:35.342618 (+ 253us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.93.190:38049: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":630,"threads_started":1}
W20260810 11:44:35.343336 26743 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260810 11:44:35.343961 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26673
I20260810 11:44:35.345393 26735 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 11:44:35.355893 26775 webserver.cc:533] Webserver: sslize failed:
E20260810 11:44:35.356014 26775 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:44:35.443926 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26673
2026-08-10T11:44:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (782 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T11:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:35Z Disabled control of system clock
I20260810 11:44:35.543576 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40523
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:40523
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/testchainca.cert with env {}
W20260810 11:44:35.894394 26780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:35.894793 26780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:35.894949 26780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:35.902374 26780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:35.902521 26780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:35.902608 26780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:35.902720 26780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:35.902798 26780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:44:35.902887 26780 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:44:35.902971 26780 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 11:44:35.913044 26780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:40523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:35.914541 26780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:35.915899 26780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:35.927946 26785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:35.927948 26788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:35.928050 26786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:35.929006 26780 server_base.cc:1034] running on GCE node
I20260810 11:44:35.929946 26780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:35.931397 26780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:35.932629 26780 hybrid_clock.cc:648] HybridClock initialized: now 1786362275932594 us; error 49 us; skew 500 ppm
I20260810 11:44:35.933004 26780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:44:35.935529 26780 webserver.cc:460] Webserver started at http://127.20.93.190:38937/ using document root <none> and password file <none>
I20260810 11:44:35.936271 26780 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:35.936410 26780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:35.936735 26780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:35.939785 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a011a5437330436dbe9d8ceec7d20ad7"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "37008aecfa29e60dd5bc1b259c5af150"
server_key_iv: "560f7cfdb039f4f0e8c2319f3b398062"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.940629 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a011a5437330436dbe9d8ceec7d20ad7"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "37008aecfa29e60dd5bc1b259c5af150"
server_key_iv: "560f7cfdb039f4f0e8c2319f3b398062"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.945995 26780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:44:35.949374 26794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:35.950870 26780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:35.951085 26780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a011a5437330436dbe9d8ceec7d20ad7"
format_stamp: "Formatted at 2026-08-10 11:44:35 on dist-test-slave-njxd"
server_key: "37008aecfa29e60dd5bc1b259c5af150"
server_key_iv: "560f7cfdb039f4f0e8c2319f3b398062"
server_key_version: "encryptionkey@0"
I20260810 11:44:35.951318 26780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:35.971232 26780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:35.972085 26780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:35.972378 26780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:35.996614 26780 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:40523
I20260810 11:44:35.996632 26850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:40523 every 8 connection(s)
I20260810 11:44:35.998077 26780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:36.000952 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26780
I20260810 11:44:36.001161 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:36.001917 20854 external_mini_cluster.cc:1428] Setting key 1d2aa0c6d003cc27ff96310fb670db7a
I20260810 11:44:36.002110 26851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:36.015985 26851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: Bootstrap starting.
I20260810 11:44:36.016115 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:36.003833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.20.93.190:40523)
0810 11:44:36.004837 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:36.004851 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:36.004893 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:44:36.006840 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:36.006863 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:36.007343 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:36.007700 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:36.008316 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.009744 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.011527 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.012190 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.012326 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:36.013726 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:36.013736 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:36.013750 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:44:36.014336 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:36.014412 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:44:36.014432 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:36.014539 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:36.014831 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 11:44:36.015185 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":359,"threads_started":1}
I20260810 11:44:36.019677 26851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:36.020164 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:44:36.020926 26851 log.cc:824] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:36.024032 26851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: No bootstrap required, opened a new log
I20260810 11:44:36.031270 26851 raft_consensus.cc:348] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } }
I20260810 11:44:36.031634 26851 raft_consensus.cc:374] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:36.031714 26851 raft_consensus.cc:729] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a011a5437330436dbe9d8ceec7d20ad7, State: Initialized, Role: FOLLOWER
I20260810 11:44:36.032269 26851 consensus_queue.cc:260] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } }
I20260810 11:44:36.032495 26851 raft_consensus.cc:388] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:36.032652 26851 raft_consensus.cc:482] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:36.032855 26851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:36.035226 26851 raft_consensus.cc:504] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } }
I20260810 11:44:36.035722 26851 leader_election.cc:302] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a011a5437330436dbe9d8ceec7d20ad7; no voters:
I20260810 11:44:36.036314 26851 leader_election.cc:288] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:36.036463 26856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:36.037249 26856 raft_consensus.cc:686] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [term 1 LEADER]: Becoming Leader. State: Replica: a011a5437330436dbe9d8ceec7d20ad7, State: Running, Role: LEADER
I20260810 11:44:36.037820 26856 consensus_queue.cc:237] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } }
I20260810 11:44:36.038769 26851 sys_catalog.cc:564] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:36.040802 26857 sys_catalog.cc:455] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a011a5437330436dbe9d8ceec7d20ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } } }
I20260810 11:44:36.041204 26857 sys_catalog.cc:458] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:36.045770 26858 sys_catalog.cc:455] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a011a5437330436dbe9d8ceec7d20ad7. Latest consensus state: current_term: 1 leader_uuid: "a011a5437330436dbe9d8ceec7d20ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a011a5437330436dbe9d8ceec7d20ad7" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40523 } } }
I20260810 11:44:36.046144 26858 sys_catalog.cc:458] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7 [sys.catalog]: This master's current role is: LEADER
W20260810 11:44:36.050310 26873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:44:36.050442 26873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:44:36.050588 26866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:36.052668 26866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:36.061264 26866 catalog_manager.cc:1347] Generated new cluster ID: 3ffbe015fc80405b9bda2ba36c70b2b8
I20260810 11:44:36.061395 26866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:36.078101 26866 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:36.079488 26866 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:36.096372 26866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a011a5437330436dbe9d8ceec7d20ad7: Generated new TSK 0
I20260810 11:44:36.097523 26866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:44:36.174108 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:36.122724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.20.93.190:40523)
0810 11:44:36.122993 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:36.123002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:36.123605 (+ 603us) server_negotiation.cc:407] Connection header received
0810 11:44:36.123884 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:36.123888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:36.123957 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:36.124119 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:44:36.124614 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.125363 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.126868 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.127333 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.127474 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:36.127988 (+ 514us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:44:36.173285 (+ 45297us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 11:44:36.173774 (+ 489us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51450: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:44:36.174270 26854 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51450: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 11:44:36.174423 26882 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:36.122619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:40523 (local address 127.0.0.1:51450)
0810 11:44:36.123491 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:36.123509 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:36.123683 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:36.124142 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:36.124147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:36.124159 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:44:36.124441 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:36.124451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.125522 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.125528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:36.126713 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:36.126726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:36.127529 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:36.127536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:36.127673 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:36.127708 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:44:36.173634 (+ 45926us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 11:44:36.173950 (+ 316us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.93.190:40523: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":587,"threads_started":1}
I20260810 11:44:36.175567 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26780
I20260810 11:44:36.176919 26846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:36.284689 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26780
2026-08-10T11:44:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (815 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (432 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (434 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:37 dist-test-slave-njxd krb5kdc[26898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:37 dist-test-slave-njxd krb5kdc[26898](info): set up 2 sockets
Aug 10 11:44:37 dist-test-slave-njxd krb5kdc[26898](info): commencing operation
krb5kdc: starting...
W20260810 11:44:38.301414 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:38 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:44:38.565668 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33605
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:33605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:38.915059 26912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:38.915462 26912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:38.915797 26912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:38.923715 26912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:38.923868 26912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:38.923918 26912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:38.924006 26912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:38.924105 26912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:38.934736 26912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:33605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:38.936308 26912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:38.937844 26912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:38.950819 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:38.950781 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:38.950727 26921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:38.951786 26912 server_base.cc:1034] running on GCE node
I20260810 11:44:38.952574 26912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:38.954154 26912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:38.955384 26912 hybrid_clock.cc:648] HybridClock initialized: now 1786362278955325 us; error 47 us; skew 500 ppm
I20260810 11:44:38.955780 26912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:38 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362278, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:38.962327 26912 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:44:38.963958 26912 webserver.cc:460] Webserver started at http://127.20.93.190:42871/ using document root <none> and password file <none>
I20260810 11:44:38.964803 26912 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:38.964936 26912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:38.965274 26912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:38.968536 26912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff"
format_stamp: "Formatted at 2026-08-10 11:44:38 on dist-test-slave-njxd"
server_key: "12b1fe051bd76abdfd6c4483e7fa020c"
server_key_iv: "f36815bbf5a7e51345101ede06b59969"
server_key_version: "encryptionkey@0"
I20260810 11:44:38.969372 26912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff"
format_stamp: "Formatted at 2026-08-10 11:44:38 on dist-test-slave-njxd"
server_key: "12b1fe051bd76abdfd6c4483e7fa020c"
server_key_iv: "f36815bbf5a7e51345101ede06b59969"
server_key_version: "encryptionkey@0"
I20260810 11:44:38.974875 26912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:44:38.978531 26928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:38.979980 26912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:38.980180 26912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff"
format_stamp: "Formatted at 2026-08-10 11:44:38 on dist-test-slave-njxd"
server_key: "12b1fe051bd76abdfd6c4483e7fa020c"
server_key_iv: "f36815bbf5a7e51345101ede06b59969"
server_key_version: "encryptionkey@0"
I20260810 11:44:38.980389 26912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:38.996069 26912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:39.000316 26912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:39.000604 26912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:39.018695 26912 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:33605
I20260810 11:44:39.018728 26981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:33605 every 8 connection(s)
I20260810 11:44:39.020141 26912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:39.021795 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 26912
I20260810 11:44:39.022023 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:39.022881 20854 external_mini_cluster.cc:1428] Setting key 389bd42f31fd4097d7466ea9cdd02826
I20260810 11:44:39.025190 26982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:44:39 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:39.039162 26982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff: Bootstrap starting.
I20260810 11:44:39.043545 26982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:39.044857 26982 log.cc:824] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:39.047881 26985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:39.024990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.20.93.190:33605)
0810 11:44:39.025792 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:39.025804 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:39.025847 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:44:39.027493 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:39.027514 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:39.027901 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:39.028239 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:39.030125 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.031480 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.033347 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.034081 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.034209 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:39.039081 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:39.039091 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:39.039111 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:39.039166 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:39.043139 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:39.044013 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:39.044024 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:39.044034 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:39.044238 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:39.044621 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:39.044628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:39.044636 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:39.045369 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:39.045721 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:39.046114 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 11:44:39.046552 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":352,"threads_started":1}
I20260810 11:44:39.048463 26982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff: No bootstrap required, opened a new log
I20260810 11:44:39.056154 26982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } }
I20260810 11:44:39.056558 26982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:39.056692 26982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 562fe3a7e5bb47eeb1d4a7ac46ca07ff, State: Initialized, Role: FOLLOWER
I20260810 11:44:39.057371 26982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } }
I20260810 11:44:39.057667 26982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:39.057824 26982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:39.058010 26982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:39.060348 26982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } }
I20260810 11:44:39.060884 26982 leader_election.cc:302] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 562fe3a7e5bb47eeb1d4a7ac46ca07ff; no voters:
I20260810 11:44:39.061551 26982 leader_election.cc:288] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:39.061769 26987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:39.062491 26987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [term 1 LEADER]: Becoming Leader. State: Replica: 562fe3a7e5bb47eeb1d4a7ac46ca07ff, State: Running, Role: LEADER
I20260810 11:44:39.063055 26987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } }
I20260810 11:44:39.063737 26982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:39.066088 26989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 562fe3a7e5bb47eeb1d4a7ac46ca07ff. Latest consensus state: current_term: 1 leader_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } } }
I20260810 11:44:39.066636 26989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:39.066152 26988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562fe3a7e5bb47eeb1d4a7ac46ca07ff" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33605 } } }
I20260810 11:44:39.066805 26988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:39.071148 26996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:39.077126 26996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:39.087133 26996 catalog_manager.cc:1347] Generated new cluster ID: 29261fbcfa1e4adea39e352e1bb91a8b
I20260810 11:44:39.087261 26996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:39.105650 26996 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:39.107005 26996 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:39.115820 26996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 562fe3a7e5bb47eeb1d4a7ac46ca07ff: Generated new TSK 0
I20260810 11:44:39.116914 26996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:44:39.183555 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:39.528996 27009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:39.529335 27009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:39.529433 27009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:39.536926 27009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:39.537055 27009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:39.537173 27009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:39.547649 27009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:39.548950 27009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:39.550324 27009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:39.563643 27014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:39.563733 27015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:39.563603 27018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:39.571156 27009 server_base.cc:1034] running on GCE node
I20260810 11:44:39.571902 27009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:39.572875 27009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:39.574110 27009 hybrid_clock.cc:648] HybridClock initialized: now 1786362279574052 us; error 51 us; skew 500 ppm
I20260810 11:44:39.574445 27009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:39 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362279, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:39.577765 27009 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:44:39.579393 27009 webserver.cc:460] Webserver started at http://127.20.93.129:45275/ using document root <none> and password file <none>
I20260810 11:44:39.580163 27009 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:39.580317 27009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:39.580638 27009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:39.583976 27009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "e63ecfabb80a4527bac3ea4f49200fa6"
format_stamp: "Formatted at 2026-08-10 11:44:39 on dist-test-slave-njxd"
server_key: "71a8aacbdd3d5899af9cb9371e3b78fd"
server_key_iv: "ebcb27d586935ef7924ab70cd3aa9fff"
server_key_version: "encryptionkey@0"
I20260810 11:44:39.584854 27009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "e63ecfabb80a4527bac3ea4f49200fa6"
format_stamp: "Formatted at 2026-08-10 11:44:39 on dist-test-slave-njxd"
server_key: "71a8aacbdd3d5899af9cb9371e3b78fd"
server_key_iv: "ebcb27d586935ef7924ab70cd3aa9fff"
server_key_version: "encryptionkey@0"
I20260810 11:44:39.590386 27009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 11:44:39.593926 27026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:39.595291 27009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:44:39.595578 27009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "e63ecfabb80a4527bac3ea4f49200fa6"
format_stamp: "Formatted at 2026-08-10 11:44:39 on dist-test-slave-njxd"
server_key: "71a8aacbdd3d5899af9cb9371e3b78fd"
server_key_iv: "ebcb27d586935ef7924ab70cd3aa9fff"
server_key_version: "encryptionkey@0"
I20260810 11:44:39.595814 27009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:39.615304 27009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:39.619777 27009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:39.620033 27009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:39.620980 27009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:39.622444 27009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:39.622519 27009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:39.622610 27009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:39.622654 27009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:39.657501 27009 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:43781
I20260810 11:44:39.657538 27146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:43781 every 8 connection(s)
I20260810 11:44:39.658959 27009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:39.659824 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27009
I20260810 11:44:39.660056 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:39.660841 20854 external_mini_cluster.cc:1428] Setting key 5b8280e1f71772b385b6931d341152d7
Aug 10 11:44:39 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362279, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:39.681926 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:39.662049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46167 (local address 127.20.93.190:33605)
0810 11:44:39.662518 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:39.662524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:39.663826 (+ 1302us) server_negotiation.cc:407] Connection header received
0810 11:44:39.666052 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:39.666061 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:39.666158 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:39.666297 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:39.668012 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.669201 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.671134 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.671883 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.672048 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:39.675998 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:39.676003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:39.676006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:39.676040 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:39.678639 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:39.679455 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:39.679461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:39.679464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:39.679601 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:39.680069 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:39.680073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:39.680075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:39.680357 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:39.680536 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:39.681274 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 11:44:39.681520 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260810 11:44:39.683030 27150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:39.662696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33605 (local address 127.20.93.129:46167)
0810 11:44:39.663629 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:39.663687 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:39.665704 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:39.666630 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:39.666639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:39.667184 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:39.667812 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:39.667832 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.669488 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.669492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:39.670994 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:39.671007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:39.672149 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:39.672159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:39.672518 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:39.673299 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:39.673346 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:39.675759 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:39.678836 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:39.678845 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:39.678864 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:39.679314 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:39.679719 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:39.679724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:39.679728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:39.679932 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:39.680493 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:39.680503 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:39.680880 (+ 377us) client_negotiation.cc:769] Sending connection context
0810 11:44:39.681161 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 11:44:39.681776 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":380,"threads_started":1}
I20260810 11:44:39.684870 27147 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33605
I20260810 11:44:39.685236 27147 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:44:39.686127 27147 heartbeater.cc:511] Master 127.20.93.190:33605 requested a full tablet report, sending...
I20260810 11:44:39.688349 26947 ts_manager.cc:194] Registered new tserver with Master: e63ecfabb80a4527bac3ea4f49200fa6 (127.20.93.129:43781)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:44:39.690074 26947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:46167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:44:39.718544 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:40.075484 27153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:40.075798 27153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:40.075894 27153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:40.083585 27153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:40.083695 27153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:40.083880 27153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:40.093982 27153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:40.095260 27153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:40.096546 27153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:40.110028 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:40.110100 27159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:40.110129 27160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:40.110806 27153 server_base.cc:1034] running on GCE node
I20260810 11:44:40.111626 27153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:40.112748 27153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:40.113974 27153 hybrid_clock.cc:648] HybridClock initialized: now 1786362280113936 us; error 50 us; skew 500 ppm
I20260810 11:44:40.114300 27153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.117434 27153 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:44:40.118933 27153 webserver.cc:460] Webserver started at http://127.20.93.130:38631/ using document root <none> and password file <none>
I20260810 11:44:40.119658 27153 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:40.119802 27153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:40.120107 27153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:40.123155 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "5bc83248058c4dff936796ba4c1ab551"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "5c72a59a3506f03db1cdf9c9e93a8191"
server_key_iv: "7afbbf5ad641437a42cd6aa7e1d1f57c"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.123978 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "5bc83248058c4dff936796ba4c1ab551"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "5c72a59a3506f03db1cdf9c9e93a8191"
server_key_iv: "7afbbf5ad641437a42cd6aa7e1d1f57c"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.129189 27153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:44:40.133172 27169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.134567 27153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:40.134745 27153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "5bc83248058c4dff936796ba4c1ab551"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "5c72a59a3506f03db1cdf9c9e93a8191"
server_key_iv: "7afbbf5ad641437a42cd6aa7e1d1f57c"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.134960 27153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:40.153370 27153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:40.157372 27153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:40.157701 27153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:40.158720 27153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:40.160090 27153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:40.160195 27153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.160348 27153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:40.160427 27153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.194978 27153 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42781
I20260810 11:44:40.195027 27285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42781 every 8 connection(s)
I20260810 11:44:40.196359 27153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:40.204559 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27153
I20260810 11:44:40.204875 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:40.205349 20854 external_mini_cluster.cc:1428] Setting key 76588fb01f2cda179be7d3e3c310abbb
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:40.217810 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.199056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:48809 (local address 127.20.93.190:33605)
0810 11:44:40.199568 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.199575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.200612 (+ 1037us) server_negotiation.cc:407] Connection header received
0810 11:44:40.203060 (+ 2448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.203065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.203140 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.203245 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.204881 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.205812 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.207444 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.207909 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.208050 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.211745 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.211750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.211753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.211786 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.214618 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.215382 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.215391 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.215395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.215531 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.215988 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.215994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.215997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.216275 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.216437 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.217153 (+ 716us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.217386 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":240,"threads_started":1}
I20260810 11:44:40.218883 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.199689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33605 (local address 127.20.93.130:48809)
0810 11:44:40.200435 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:40.200475 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:40.202770 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:40.203478 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:40.203486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:40.204039 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:40.204677 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:40.204695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.205982 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.205987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:40.207285 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:40.207298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.208156 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.208164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:40.208416 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.209142 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:40.209170 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:40.211481 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:40.214779 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:40.214788 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:40.214807 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:40.215252 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:40.215649 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:40.215654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:40.215658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:40.215869 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:40.216408 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:40.216418 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:40.216790 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 11:44:40.217076 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 11:44:40.217688 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":277,"threads_started":1}
I20260810 11:44:40.220460 27286 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33605
I20260810 11:44:40.220779 27286 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:40.221547 27286 heartbeater.cc:511] Master 127.20.93.190:33605 requested a full tablet report, sending...
I20260810 11:44:40.223173 26947 ts_manager.cc:194] Registered new tserver with Master: 5bc83248058c4dff936796ba4c1ab551 (127.20.93.130:42781)
I20260810 11:44:40.224478 26947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:48809
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:44:40.261445 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:40.606351 27292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:40.606662 27292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:40.606856 27292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:40.614279 27292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:40.614403 27292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:40.614627 27292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:40.624656 27292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:40.626226 27292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:40.627596 27292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:40.641131 27297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:40.641263 27298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:40.641768 27300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:40.642365 27292 server_base.cc:1034] running on GCE node
I20260810 11:44:40.642997 27292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:40.644001 27292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:40.645234 27292 hybrid_clock.cc:648] HybridClock initialized: now 1786362280645172 us; error 61 us; skew 500 ppm
I20260810 11:44:40.645617 27292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.648880 27292 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:44:40.650509 27292 webserver.cc:460] Webserver started at http://127.20.93.131:45367/ using document root <none> and password file <none>
I20260810 11:44:40.651243 27292 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:40.651398 27292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:40.651726 27292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:40.654875 27292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "3f2febe0a706426a8e3298d69be84323"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "0032732912fc3da9b506683cedb97402"
server_key_iv: "0e07cb49fa1b20765af59d543c363917"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.655727 27292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "3f2febe0a706426a8e3298d69be84323"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "0032732912fc3da9b506683cedb97402"
server_key_iv: "0e07cb49fa1b20765af59d543c363917"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.662791 27292 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260810 11:44:40.666951 27307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.668568 27292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 11:44:40.668720 27292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "3f2febe0a706426a8e3298d69be84323"
format_stamp: "Formatted at 2026-08-10 11:44:40 on dist-test-slave-njxd"
server_key: "0032732912fc3da9b506683cedb97402"
server_key_iv: "0e07cb49fa1b20765af59d543c363917"
server_key_version: "encryptionkey@0"
I20260810 11:44:40.668921 27292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:40.685068 27292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:40.689649 27292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:40.689905 27292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:40.690861 27292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:40.692231 27292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:40.692306 27292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.692396 27292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:40.692440 27292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:40.693092 27147 heartbeater.cc:503] Master 127.20.93.190:33605 was elected leader, sending a full tablet report...
I20260810 11:44:40.727298 27292 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:33403
I20260810 11:44:40.727332 27427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:33403 every 8 connection(s)
I20260810 11:44:40.728782 27292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:40.738117 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27292
I20260810 11:44:40.738343 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:40.738929 20854 external_mini_cluster.cc:1428] Setting key 2a18590338d617839f2c4216c7935e28
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:44:40.750340 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.731680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:43223 (local address 127.20.93.190:33605)
0810 11:44:40.732157 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.732162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.733160 (+ 998us) server_negotiation.cc:407] Connection header received
0810 11:44:40.735119 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.735124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.735237 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.735360 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.736936 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.737819 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.739809 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.740396 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.740516 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.744411 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.744416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.744420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.744454 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.747332 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.748097 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.748101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.748104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.748232 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.748672 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.748679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.748686 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.748975 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.749113 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.749880 (+ 767us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.750061 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
I20260810 11:44:40.751386 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.732204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33605 (local address 127.20.93.131:43223)
0810 11:44:40.732976 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:40.733015 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:40.734825 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:40.735616 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:40.735626 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:40.736167 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:40.736754 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:40.736772 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.737993 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.737997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:40.739623 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:40.739640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.740666 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.740673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:40.740896 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.741728 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:40.741752 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:40.744199 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:40.747484 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:40.747492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:40.747508 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:40.747973 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:40.748349 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:40.748353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:40.748355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:40.748551 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:40.749095 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:40.749102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:40.749447 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 11:44:40.749785 (+ 338us) client_negotiation.cc:240] Negotiation successful
0810 11:44:40.750210 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":305,"threads_started":1}
I20260810 11:44:40.752871 27428 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33605
I20260810 11:44:40.753324 27428 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:40.754298 27428 heartbeater.cc:511] Master 127.20.93.190:33605 requested a full tablet report, sending...
I20260810 11:44:40.755805 26947 ts_manager.cc:194] Registered new tserver with Master: 3f2febe0a706426a8e3298d69be84323 (127.20.93.131:33403)
I20260810 11:44:40.756726 26947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:43223
I20260810 11:44:40.767513 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/test-admin.keytab.
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.799552 20854 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 11:44:40.805063 27436 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:40.802396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.20.93.129:43781)
0810 11:44:40.803115 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.803122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.803147 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:44:40.803208 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.803215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.803427 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.803607 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.804597 (+ 990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40752: BlockingRecv error: recv got EOF from 127.0.0.1:40752 (error 108)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":352,"threads_started":1}
W20260810 11:44:40.805203 20854 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.807016 20854 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.808398 20854 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:40.824427 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.810088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.20.93.129:43781)
0810 11:44:40.810302 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.810310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.810649 (+ 339us) server_negotiation.cc:407] Connection header received
0810 11:44:40.810848 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.810853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.810943 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.811050 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.812050 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.813067 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.814405 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.815009 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.815155 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.818865 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.818884 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.818899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.818945 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.821672 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.822319 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.822328 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.822331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.822469 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.822852 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.822857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.822861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.823223 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.823524 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.823851 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.824103 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 11:44:40.830564 27436 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:40.828648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.20.93.129:43781)
0810 11:44:40.828842 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.828850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.829155 (+ 305us) server_negotiation.cc:407] Connection header received
0810 11:44:40.829362 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.829365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.829423 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.829524 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.830307 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40758: BlockingRecv error: recv got EOF from 127.0.0.1:40758 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 11:44:40.830760 20854 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.832458 20854 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.833917 20854 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:40.847855 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.835406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.20.93.129:43781)
0810 11:44:40.835622 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.835628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.835887 (+ 259us) server_negotiation.cc:407] Connection header received
0810 11:44:40.836013 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.836016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.836074 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.836153 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.837063 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.837807 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.839031 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.839448 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.839542 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.843071 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.843075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.843079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.843111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.845542 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.846197 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.846201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.846203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.846340 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.846657 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.846660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.846662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.846914 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.847114 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.847345 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.847543 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 11:44:40.852216 27436 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:40.850673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.20.93.129:43781)
0810 11:44:40.850866 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.850879 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.850979 (+ 100us) server_negotiation.cc:407] Connection header received
0810 11:44:40.851112 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.851115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.851185 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.851265 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.851946 (+ 681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40770: BlockingRecv error: recv got EOF from 127.0.0.1:40770 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 11:44:40.852372 20854 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786362228785499-20854-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.853991 20854 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:40.855330 20854 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:44:40.869683 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.856804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.20.93.129:43781)
0810 11:44:40.857017 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.857022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.857118 (+ 96us) server_negotiation.cc:407] Connection header received
0810 11:44:40.857251 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.857254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.857310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.857386 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.858359 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.859077 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.860264 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.860676 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.860777 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.864252 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.864256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.864260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.864332 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.867010 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.867601 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.867605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.867607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.867788 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.868193 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.868199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.868202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.868492 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.868822 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.869070 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.869288 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:44:40.871362 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27009
I20260810 11:44:40.883059 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.872111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.20.93.129:43781)
0810 11:44:40.872269 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.872274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.872649 (+ 375us) server_negotiation.cc:407] Connection header received
0810 11:44:40.872887 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.872890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.872943 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.873035 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.874056 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.874779 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.875854 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.876256 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.876347 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.878331 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.878335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.878338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.878366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:40.880659 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.881211 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.881222 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.881224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.881343 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:40.881922 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:40.881925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:40.881928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:40.882167 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:40.882377 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:40.882636 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 11:44:40.882794 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 11:44:40.883675 27142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:40.969244 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27009
I20260810 11:44:40.988332 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27153
Aug 10 11:44:40 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:44:41.006608 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:40.989384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.20.93.130:42781)
0810 11:44:40.990106 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:40.990121 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:40.990148 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:44:40.990232 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:40.990239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:40.990464 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:40.990606 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:40.991627 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.992704 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.994337 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:40.994941 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:40.995046 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:40.999395 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:40.999409 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:40.999425 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:40.999472 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:41.003342 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:41.004145 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:41.004155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:41.004158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:41.004313 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:41.004772 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:41.004778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:41.004781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:41.005162 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:41.005498 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:41.005931 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 11:44:41.006214 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":403,"threads_started":1}
I20260810 11:44:41.008010 27281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:41.096481 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27153
I20260810 11:44:41.118073 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27292
Aug 10 11:44:41 dist-test-slave-njxd krb5kdc[26898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:44:41.135541 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:41.119228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.20.93.131:33403)
0810 11:44:41.119883 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:41.119890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:41.119919 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:44:41.119984 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:41.119990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:41.120213 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:41.120396 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:41.121669 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:41.122708 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:41.124029 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:41.124625 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:41.124776 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:41.129308 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:41.129322 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:41.129337 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:41.129382 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:41.132596 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:41.133375 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:41.133383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:41.133385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:41.133514 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:41.133981 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:41.133984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:41.133986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:41.134374 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:41.134630 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:41.134974 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 11:44:41.135199 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":347,"threads_started":1}
I20260810 11:44:41.136988 27423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:41.222230 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27292
I20260810 11:44:41.241981 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 26912
I20260810 11:44:41.243347 26977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:41.351046 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 26912
2026-08-10T11:44:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4210 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:41 dist-test-slave-njxd krb5kdc[27458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:41 dist-test-slave-njxd krb5kdc[27458](info): set up 2 sockets
Aug 10 11:44:41 dist-test-slave-njxd krb5kdc[27458](info): commencing operation
krb5kdc: starting...
W20260810 11:44:42.518064 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:42 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab.
Entry for principal oryx/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab.
I20260810 11:44:42.774098 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34189
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab
--principal=oryx/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:34189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:43.114117 27472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:43.114442 27472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:43.114557 27472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:43.122500 27472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:43.122627 27472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:43.122681 27472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:43.122738 27472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:43.122782 27472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:43.132715 27472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:34189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab
--principal=oryx/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:43.134085 27472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:43.135391 27472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:43.147313 27477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:43.147361 27479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:43.147385 27481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:43.148284 27472 server_base.cc:1034] running on GCE node
I20260810 11:44:43.149108 27472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:43.150664 27472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:43.151902 27472 hybrid_clock.cc:648] HybridClock initialized: now 1786362283151854 us; error 38 us; skew 500 ppm
I20260810 11:44:43.152262 27472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:43 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362283, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:43.155448 27472 init.cc:377] Logged in from keytab as oryx/127.20.93.190@KRBTEST.COM (short username oryx)
I20260810 11:44:43.157048 27472 webserver.cc:460] Webserver started at http://127.20.93.190:37501/ using document root <none> and password file <none>
I20260810 11:44:43.157859 27472 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:43.158028 27472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:43.158335 27472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:43.161381 27472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "144d9109aa714068a117b47cf32e9fca"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "4f3f9c58408580e6e1e72bd055f16dc9"
server_key_iv: "0600ea83b0b8e80664960ac73075753f"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.162271 27472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "144d9109aa714068a117b47cf32e9fca"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "4f3f9c58408580e6e1e72bd055f16dc9"
server_key_iv: "0600ea83b0b8e80664960ac73075753f"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.167665 27472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:43.171267 27488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:43.172693 27472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:43.172906 27472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "144d9109aa714068a117b47cf32e9fca"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "4f3f9c58408580e6e1e72bd055f16dc9"
server_key_iv: "0600ea83b0b8e80664960ac73075753f"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.173118 27472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:43.185369 27472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:43.189720 27472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:43.190019 27472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:43.208686 27472 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:34189
I20260810 11:44:43.208741 27542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:34189 every 8 connection(s)
I20260810 11:44:43.210242 27472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:43.214367 27543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:43.219547 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27472
I20260810 11:44:43.219760 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:43.220467 20854 external_mini_cluster.cc:1428] Setting key 6515b6726aafaacccbcd01fa7fdb47e3
I20260810 11:44:43.225229 27543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Bootstrap starting.
I20260810 11:44:43.228935 27543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:43.230202 27543 log.cc:824] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:43.233510 27543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: No bootstrap required, opened a new log
Aug 10 11:44:43 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:43.244361 27543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } }
I20260810 11:44:43.244822 27543 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:43.244956 27543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144d9109aa714068a117b47cf32e9fca, State: Initialized, Role: FOLLOWER
I20260810 11:44:43.245688 27543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } }
I20260810 11:44:43.245929 27547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:43.222574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.20.93.190:34189)
0810 11:44:43.223468 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:43.223480 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:43.223529 (+ 49us) server_negotiation.cc:407] Connection header received
0810 11:44:43.225086 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:43.225107 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:43.225553 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:43.225926 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:43.227761 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.229266 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.231016 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.231905 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.232034 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:43.236896 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:43.236906 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:43.236926 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:43.236973 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:43.241089 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:43.241833 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:43.241843 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:43.241868 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:43.242094 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:43.242495 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:43.242500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:43.242504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:43.243291 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:43.243608 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:43.243952 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 11:44:43.244455 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":444,"threads_started":1}
I20260810 11:44:43.245982 27543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:43.246109 27543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:43.246315 27543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:43.248695 27543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } }
I20260810 11:44:43.249207 27543 leader_election.cc:302] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144d9109aa714068a117b47cf32e9fca; no voters:
I20260810 11:44:43.249919 27543 leader_election.cc:288] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:43.250036 27549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:43.250761 27549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 LEADER]: Becoming Leader. State: Replica: 144d9109aa714068a117b47cf32e9fca, State: Running, Role: LEADER
I20260810 11:44:43.251253 27549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } }
I20260810 11:44:43.251969 27543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:43.254489 27551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144d9109aa714068a117b47cf32e9fca. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } }
I20260810 11:44:43.254956 27551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:43.256326 27550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } }
I20260810 11:44:43.256661 27550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:43.258891 27556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:43.265590 27556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:43.276532 27556 catalog_manager.cc:1347] Generated new cluster ID: 04def2b6653248a2b8f356722d9a7daa
I20260810 11:44:43.276734 27556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:43.291069 27556 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:43.293066 27556 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:43.307451 27556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Generated new TSK 0
I20260810 11:44:43.308597 27556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
I20260810 11:44:43.373764 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:43.712944 27572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:43.713296 27572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:43.713470 27572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:43.720736 27572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:43.720922 27572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:43.721109 27572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:43.730737 27572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:43.732053 27572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:43.733434 27572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:43.746954 27580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:43.746986 27578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:43.747426 27577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:43.748239 27572 server_base.cc:1034] running on GCE node
I20260810 11:44:43.748903 27572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:43.749945 27572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:43.751228 27572 hybrid_clock.cc:648] HybridClock initialized: now 1786362283751078 us; error 105 us; skew 500 ppm
I20260810 11:44:43.751678 27572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:43 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362283, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:43.756457 27572 init.cc:377] Logged in from keytab as oryx/127.20.93.129@KRBTEST.COM (short username oryx)
I20260810 11:44:43.759717 27572 webserver.cc:460] Webserver started at http://127.20.93.129:41429/ using document root <none> and password file <none>
I20260810 11:44:43.760708 27572 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:43.760866 27572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:43.761194 27572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:43.764356 27572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "4dc17f0a5c6043b8a6131b6641f19ef0"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "1e6ec87f7fe1096466edf8ee555c3437"
server_key_iv: "d3e3b340a98f48cac36d98702dab774e"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.765247 27572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "4dc17f0a5c6043b8a6131b6641f19ef0"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "1e6ec87f7fe1096466edf8ee555c3437"
server_key_iv: "d3e3b340a98f48cac36d98702dab774e"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.770960 27572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 11:44:43.774379 27588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:43.775923 27572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:43.776090 27572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "4dc17f0a5c6043b8a6131b6641f19ef0"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "1e6ec87f7fe1096466edf8ee555c3437"
server_key_iv: "d3e3b340a98f48cac36d98702dab774e"
server_key_version: "encryptionkey@0"
I20260810 11:44:43.776300 27572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:43.799971 27572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:43.804157 27572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:43.804476 27572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:43.805500 27572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:43.806993 27572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:43.807070 27572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:43.807230 27572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:43.807296 27572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:43.840966 27572 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:44303
I20260810 11:44:43.841075 27708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:44303 every 8 connection(s)
I20260810 11:44:43.842427 27572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:43.851594 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27572
I20260810 11:44:43.851888 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:43.852665 20854 external_mini_cluster.cc:1428] Setting key 3444e25555cb234e4cc7d2c47f761e1d
Aug 10 11:44:43 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362283, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:43.862861 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:43.845319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:60933 (local address 127.20.93.190:34189)
0810 11:44:43.845844 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:43.845849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:43.846690 (+ 841us) server_negotiation.cc:407] Connection header received
0810 11:44:43.848515 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:43.848522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:43.848622 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:43.848747 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:43.850268 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.850985 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.852740 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.853214 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.853358 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:43.856891 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:43.856896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:43.856899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:43.856929 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:43.859668 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:43.860411 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:43.860421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:43.860423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:43.860541 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:43.860991 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:43.861000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:43.861002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:43.861256 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:43.861415 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:43.862240 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 11:44:43.862462 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":315,"threads_started":1}
I20260810 11:44:43.864012 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:43.845811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:60933)
0810 11:44:43.846524 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:43.846562 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:43.848232 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:43.848988 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:43.848996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:43.849502 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:43.850094 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:43.850112 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.851188 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.851192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:43.852585 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:43.852599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:43.853447 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:43.853457 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:43.853761 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:43.854448 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:43.854469 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:43.856647 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:43.859810 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:43.859820 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:43.859837 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:43.860278 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:43.860668 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:43.860674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:43.860678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:43.860866 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:43.861396 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:43.861404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:43.861852 (+ 448us) client_negotiation.cc:769] Sending connection context
0810 11:44:43.862127 (+ 275us) client_negotiation.cc:240] Negotiation successful
0810 11:44:43.862653 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":277,"threads_started":1}
I20260810 11:44:43.865653 27709 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:43.865988 27709 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:43.866859 27709 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:43.869280 27506 ts_manager.cc:194] Registered new tserver with Master: 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303)
I20260810 11:44:43.871600 27506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:60933
WARNING: no policy specified for oryx/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
I20260810 11:44:43.912480 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:44.262666 27717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:44.262970 27717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:44.263067 27717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:44.270750 27717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:44.270874 27717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:44.270987 27717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:44.282049 27717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:44.283362 27717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:44.284720 27717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:44.298398 27723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:44.298765 27724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:44.298993 27726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:44.300042 27717 server_base.cc:1034] running on GCE node
I20260810 11:44:44.300746 27717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:44.301770 27717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:44.302999 27717 hybrid_clock.cc:648] HybridClock initialized: now 1786362284302940 us; error 70 us; skew 500 ppm
I20260810 11:44:44.303324 27717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:44 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362284, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:44.310567 27717 init.cc:377] Logged in from keytab as oryx/127.20.93.130@KRBTEST.COM (short username oryx)
I20260810 11:44:44.312179 27717 webserver.cc:460] Webserver started at http://127.20.93.130:36415/ using document root <none> and password file <none>
I20260810 11:44:44.312948 27717 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:44.313103 27717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:44.313426 27717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:44.316587 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "3408f93054024ae38fdb58991acd437b"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "5d7c8af60928f5d2039b605fcb7302cd"
server_key_iv: "019c7d023211321c14fc2d1aa6e35b89"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.317482 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "3408f93054024ae38fdb58991acd437b"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "5d7c8af60928f5d2039b605fcb7302cd"
server_key_iv: "019c7d023211321c14fc2d1aa6e35b89"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.322845 27717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:44:44.326354 27733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.327718 27717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:44.327901 27717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "3408f93054024ae38fdb58991acd437b"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "5d7c8af60928f5d2039b605fcb7302cd"
server_key_iv: "019c7d023211321c14fc2d1aa6e35b89"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.328102 27717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:44.347676 27717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:44.352038 27717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:44.352407 27717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:44.353660 27717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:44.355094 27717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:44.355204 27717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.355340 27717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:44.355418 27717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.389750 27717 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44411
I20260810 11:44:44.389783 27855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44411 every 8 connection(s)
I20260810 11:44:44.391134 27717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:44.399322 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27717
I20260810 11:44:44.399546 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:44.400156 20854 external_mini_cluster.cc:1428] Setting key 7756a0dc2302dff829b14a75e15928e7
Aug 10 11:44:44 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362284, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:44.413921 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:44.393902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:60771 (local address 127.20.93.190:34189)
0810 11:44:44.394386 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:44.394392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:44.395478 (+ 1086us) server_negotiation.cc:407] Connection header received
0810 11:44:44.397352 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:44.397356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:44.397426 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:44.397529 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:44.399185 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.400344 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.402741 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.403394 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.403551 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:44.407502 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:44.407507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:44.407510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:44.407543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:44.410250 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:44.411192 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:44.411195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:44.411198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:44.411333 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:44.411823 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:44.411826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:44.411828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:44.412080 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:44.412191 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:44.413328 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0810 11:44:44.413505 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260810 11:44:44.415151 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:44.394525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:60771)
0810 11:44:44.395290 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:44.395338 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:44.397077 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:44.397820 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:44.397830 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:44.398389 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:44.398980 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:44.398999 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.400518 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.400522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:44.402578 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:44.402599 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.403610 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.403614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:44.403826 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:44.404552 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:44.404581 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:44.407237 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:44.410393 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:44.410404 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:44.410424 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:44.411072 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:44.411441 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:44.411449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:44.411455 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:44.411706 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:44.412217 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:44.412227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:44.412836 (+ 609us) client_negotiation.cc:769] Sending connection context
0810 11:44:44.413246 (+ 410us) client_negotiation.cc:240] Negotiation successful
0810 11:44:44.413948 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":289,"threads_started":1}
I20260810 11:44:44.416818 27856 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:44.417153 27856 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:44.417990 27856 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:44.419557 27506 ts_manager.cc:194] Registered new tserver with Master: 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
I20260810 11:44:44.420403 27506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:60771
WARNING: no policy specified for oryx/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
I20260810 11:44:44.461428 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:44.813086 27864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:44.813378 27864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:44.813560 27864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:44.822083 27864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:44.822199 27864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:44.822409 27864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:44.832823 27864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:44.834237 27864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:44.835629 27864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:44.853363 27870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:44.854285 27869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:44.857923 27872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:44.859808 27864 server_base.cc:1034] running on GCE node
I20260810 11:44:44.860600 27864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:44.861743 27864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:44.862986 27864 hybrid_clock.cc:648] HybridClock initialized: now 1786362284862922 us; error 65 us; skew 500 ppm
I20260810 11:44:44.863350 27864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:44 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362284, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:44.866847 27864 init.cc:377] Logged in from keytab as oryx/127.20.93.131@KRBTEST.COM (short username oryx)
I20260810 11:44:44.868428 27864 webserver.cc:460] Webserver started at http://127.20.93.131:44899/ using document root <none> and password file <none>
I20260810 11:44:44.869221 27864 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:44.869376 27864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:44.869808 27864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:44.872906 27864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "13b0558dedac4a24bba11bcf72a51030"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "a8146dc480b5aed3e832d2fafd4e0146"
server_key_iv: "3f87eaaef70b7e3d3e342410d07f1a43"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.873816 27864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "13b0558dedac4a24bba11bcf72a51030"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "a8146dc480b5aed3e832d2fafd4e0146"
server_key_iv: "3f87eaaef70b7e3d3e342410d07f1a43"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.875416 27709 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
I20260810 11:44:44.879601 27864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:44.883271 27880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.884573 27864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:44.884748 27864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "13b0558dedac4a24bba11bcf72a51030"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "a8146dc480b5aed3e832d2fafd4e0146"
server_key_iv: "3f87eaaef70b7e3d3e342410d07f1a43"
server_key_version: "encryptionkey@0"
I20260810 11:44:44.884956 27864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:44.899329 27864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:44.903689 27864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:44.903983 27864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:44.904937 27864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:44.906325 27864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:44.906431 27864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.906579 27864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:44.906665 27864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:44.940425 27864 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:39623
I20260810 11:44:44.940469 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:39623 every 8 connection(s)
I20260810 11:44:44.941952 27864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:44.948694 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 27864
I20260810 11:44:44.948901 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:44.949376 20854 external_mini_cluster.cc:1428] Setting key 823e47eeaa9f84f9c218f8d0d7642b6c
Aug 10 11:44:44 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362284, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:44.962591 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:44.944603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:47209 (local address 127.20.93.190:34189)
0810 11:44:44.945050 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:44.945055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:44.946213 (+ 1158us) server_negotiation.cc:407] Connection header received
0810 11:44:44.948052 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:44.948058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:44.948158 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:44.948287 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:44.949969 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.950965 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.952904 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.953299 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.953429 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:44.956830 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:44.956835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:44.956839 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:44.956869 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:44.959459 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:44.960195 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:44.960202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:44.960211 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:44.960326 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:44.960771 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:44.960776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:44.960779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:44.961049 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:44.961214 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:44.961975 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 11:44:44.962198 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":197,"threads_started":1}
I20260810 11:44:44.963698 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:44.945271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:47209)
0810 11:44:44.946065 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:44.946102 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:44.947785 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:44.948514 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:44.948522 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:44.949123 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:44.949803 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:44.949823 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.951117 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.951121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:44.952757 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:44.952775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:44.953533 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:44.953540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:44.953819 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:44.954434 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:44.954455 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:44.956606 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:44.959609 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:44.959618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:44.959636 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:44.960062 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:44.960438 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:44.960443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:44.960447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:44.960643 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:44.961181 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:44.961188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:44.961534 (+ 346us) client_negotiation.cc:769] Sending connection context
0810 11:44:44.961866 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 11:44:44.962431 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":361,"threads_started":1}
I20260810 11:44:44.965277 28000 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:44.965638 28000 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:44.966463 28000 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:44.968005 27505 ts_manager.cc:194] Registered new tserver with Master: 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:44.968863 27505 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:47209
I20260810 11:44:44.977552 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.93.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.189.keytab.
Entry for principal oryx/127.20.93.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.189.keytab.
I20260810 11:44:45.036626 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:38591
--webserver_interface=127.20.93.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.189.keytab
--principal=oryx/127.20.93.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:34189,127.20.93.189:38591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:45.384953 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:45.385277 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:45.385470 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:45.393703 28007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:45.393853 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:45.393950 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:45.394039 28007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:45.394129 28007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:45.404399 28007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:34189,127.20.93.189:38591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.189.keytab
--principal=oryx/127.20.93.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:38591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.93.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
W20260810 11:44:45.404843 28007 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.93.190:34189,127.20.93.189:38591'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 11:44:45.406046 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:45.407418 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:45.419588 28015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:45.419509 28017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:45.419559 28014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:45.420161 28007 server_base.cc:1034] running on GCE node
I20260810 11:44:45.420975 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:45.422492 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:45.423144 27856 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
I20260810 11:44:45.423770 28007 hybrid_clock.cc:648] HybridClock initialized: now 1786362285423682 us; error 45 us; skew 500 ppm
I20260810 11:44:45.424136 28007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362285, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:45.427461 28007 init.cc:377] Logged in from keytab as oryx/127.20.93.189@KRBTEST.COM (short username oryx)
I20260810 11:44:45.428915 28007 webserver.cc:460] Webserver started at http://127.20.93.189:43575/ using document root <none> and password file <none>
I20260810 11:44:45.429723 28007 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:45.429859 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:45.430188 28007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:45.433419 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/instance:
uuid: "0a8032ecd65c4c55b0ee3609108078e4"
format_stamp: "Formatted at 2026-08-10 11:44:45 on dist-test-slave-njxd"
server_key: "e95c5adcfba348ca147ed27f82014055"
server_key_iv: "a374420dd73320c8886573dd3049a6b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:45.434293 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal/instance:
uuid: "0a8032ecd65c4c55b0ee3609108078e4"
format_stamp: "Formatted at 2026-08-10 11:44:45 on dist-test-slave-njxd"
server_key: "e95c5adcfba348ca147ed27f82014055"
server_key_iv: "a374420dd73320c8886573dd3049a6b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:45.439891 28007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:44:45.443421 28024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:45.444818 28007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:45.445008 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "0a8032ecd65c4c55b0ee3609108078e4"
format_stamp: "Formatted at 2026-08-10 11:44:45 on dist-test-slave-njxd"
server_key: "e95c5adcfba348ca147ed27f82014055"
server_key_iv: "a374420dd73320c8886573dd3049a6b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:45.445235 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:45.457971 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:45.462247 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:45.462532 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:45.482227 28007 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.189:38591
I20260810 11:44:45.482249 28080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.189:38591 every 8 connection(s)
I20260810 11:44:45.483783 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
I20260810 11:44:45.487795 28081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:45.493261 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 28007
I20260810 11:44:45.495668 28081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362285, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.189@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
I20260810 11:44:45.515199 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.498633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.20.93.190:34189)
0810 11:44:45.499101 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.499107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.499941 (+ 834us) server_negotiation.cc:407] Connection header received
0810 11:44:45.500040 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.500047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.500158 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.500286 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.501639 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.502791 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.504394 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.504812 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.504950 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.508586 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.508591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.508594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.508630 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.511639 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.512678 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.512683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.512686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.512842 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.513251 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.513256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.513259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.513745 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.513918 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.514602 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.514840 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":240,"threads_started":1}
I20260810 11:44:45.516324 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.495800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.20.93.189:38591)
0810 11:44:45.496700 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.496725 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.496768 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:44:45.498762 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.498809 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.499317 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.499721 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.501365 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.503087 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.504339 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.504849 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.505053 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.509351 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.509359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.509370 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.509398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.512500 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.513371 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.513377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.513379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.513490 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.513981 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.513985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.513987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.514356 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.514692 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.515033 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.515282 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"spinlock_wait_cycles":17280,"thread_start_us":440,"threads_started":1}
I20260810 11:44:45.516372 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.498662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53328)
0810 11:44:45.499626 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.499667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.499825 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.500477 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.500482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.501053 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.501442 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.501456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.502939 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.502943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.504242 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.504254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.505046 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.505054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.505212 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.505976 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.505998 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.508362 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.511784 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.511795 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.511808 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.512566 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.512951 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.512954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.512956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.513122 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.513904 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.513917 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.514230 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.514520 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.515053 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":578,"threads_started":1}
I20260810 11:44:45.518165 28081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362285, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.189@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
I20260810 11:44:45.530218 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.519011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39342)
0810 11:44:45.519194 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.519216 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.519326 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.519669 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.519672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.520005 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.520131 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.520139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.521061 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.521064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.522016 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.522029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.522807 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.522812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.522941 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.523447 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.523464 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.525527 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.528131 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.528135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.528138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.528528 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.528852 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.528856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.528858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.528982 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.529423 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.529426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.529545 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.529713 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.529928 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 11:44:45.530313 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.518913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.20.93.189:38591)
0810 11:44:45.519106 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.519111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.519286 (+ 175us) server_negotiation.cc:407] Connection header received
0810 11:44:45.519448 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.519453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.519546 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.519648 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.520246 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.520927 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.522142 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.522668 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.522773 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.525743 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.525747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.525751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.525779 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.528004 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.528626 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.528630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.528632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.528756 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.529070 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.529073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.529075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.529328 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.529433 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.529788 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.530043 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":60}
I20260810 11:44:45.535342 28081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Bootstrap starting.
I20260810 11:44:45.538585 28081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:45.539654 28081 log.cc:824] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:45.542420 28081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: No bootstrap required, opened a new log
I20260810 11:44:45.549898 28081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } }
I20260810 11:44:45.550261 28081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:45.550360 28081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8032ecd65c4c55b0ee3609108078e4, State: Initialized, Role: FOLLOWER
I20260810 11:44:45.550942 28081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } }
I20260810 11:44:45.551820 28088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } } }
I20260810 11:44:45.552156 28088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:45.552973 28081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:44:45.561774 28100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:44:45.561923 28100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:44:45.564867 28007 master_runner.cc:428] Detected that this master 0a8032ecd65c4c55b0ee3609108078e4 is joining an existing cluster
I20260810 11:44:45.564980 28007 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.93.189:38591
I20260810 11:44:45.592109 28088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:45.592437 28088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } }
I20260810 11:44:45.593206 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.569274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39350)
0810 11:44:45.576690 (+ 7416us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.576726 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.576901 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.577465 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.577472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.578284 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.578757 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.578773 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.580322 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.580331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.582103 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.582120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.583522 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.583530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.583759 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.584473 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.584499 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.585762 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.589142 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.589149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.589157 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.589936 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.590403 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.590408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.590412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.590585 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.591423 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.591430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.591854 (+ 424us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.592467 (+ 613us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.592736 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7250,"thread_start_us":20278,"threads_started":1}
I20260810 11:44:45.593854 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.569777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.20.93.189:38591)
0810 11:44:45.569922 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.569928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.576803 (+ 6875us) server_negotiation.cc:407] Connection header received
0810 11:44:45.577037 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.577045 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.577118 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.577187 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.578908 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.580071 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.582250 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.583091 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.583200 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.585935 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.585941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.585946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.585982 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.589006 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.590065 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.590072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.590079 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.590260 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.590698 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.590705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.590708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.591195 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.591359 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.592144 (+ 785us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.592407 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 11:44:45.594089 28088 leader_election.cc:288] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189)
I20260810 11:44:45.595108 27516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a8032ecd65c4c55b0ee3609108078e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144d9109aa714068a117b47cf32e9fca" is_pre_election: true
I20260810 11:44:45.596465 28028 leader_election.cc:302] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 0a8032ecd65c4c55b0ee3609108078e4; no voters: 144d9109aa714068a117b47cf32e9fca
I20260810 11:44:45.597366 28088 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
WARNING: no policy specified for oryx/127.20.93.188@KRBTEST.COM; defaulting to no policy
I20260810 11:44:45.606999 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.569001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.20.93.190:34189)
0810 11:44:45.569208 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.569215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.590175 (+ 20960us) server_negotiation.cc:407] Connection header received
0810 11:44:45.590363 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.590370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.590502 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.590601 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.595620 (+ 5019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.596717 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.598846 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.599273 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.599425 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.601266 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.601271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.601275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.601322 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.604151 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.605017 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.605021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.605023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.605155 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.605654 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.605657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.605659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.605930 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.606089 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.606568 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.606758 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:44:45.607312 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.569284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53332)
0810 11:44:45.590030 (+ 20746us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.590058 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.590208 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.590816 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.590822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.591479 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.591832 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.591846 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.597117 (+ 5271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.597123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.598647 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.598659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.599467 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.599474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.599645 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.600282 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.600300 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.601062 (+ 762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.604341 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.604347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.604352 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.604887 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.605289 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.605294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.605298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.605478 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.606081 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.606086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.606307 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.606488 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.606843 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20540,"mutex_wait_us":57,"thread_start_us":6787,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.188@KRBTEST.COM" created.
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362283, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.190@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
I20260810 11:44:45.636756 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.615144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39364)
0810 11:44:45.618243 (+ 3099us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.618286 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.618635 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.619181 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.619191 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.619876 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.620118 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.620127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.621735 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.621743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.623516 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.623533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.624811 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.624818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.625033 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.625865 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.625890 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.629166 (+ 3276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.633681 (+ 4515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.633700 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.633713 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.634401 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.634888 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.634892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.634894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.635035 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.635725 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.635733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.635912 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.636090 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.636393 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2791,"thread_start_us":2618,"threads_started":1}
I20260810 11:44:45.636852 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.615184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.20.93.189:38591)
0810 11:44:45.615388 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.615397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.618434 (+ 3037us) server_negotiation.cc:407] Connection header received
0810 11:44:45.618879 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.618883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.618994 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.619113 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.620284 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.621552 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.623683 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.624613 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.624763 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.629412 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.629421 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.629426 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.629467 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.633476 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.634566 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.634572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.634576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.634757 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.635148 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.635154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.635157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.635549 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.635704 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.636218 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.636483 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:44:45.639190 27505 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.93.189:38591: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4"
member_type: NON_VOTER
last_known_addr {
host: "127.20.93.189"
port: 38591
}
attrs {
promote: true
}
}
dest_uuid: "144d9109aa714068a117b47cf32e9fca"
cas_config_opid_index: -1
I20260810 11:44:45.640177 27505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } }
I20260810 11:44:45.643528 28113 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a8032ecd65c4c55b0ee3609108078e4 (127.20.93.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.188.keytab.
Entry for principal oryx/127.20.93.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.188.keytab.
I20260810 11:44:45.644894 28053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:45.647905 28113 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.93.189:38591
I20260810 11:44:45.648568 28113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } }
I20260810 11:44:45.648904 28113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:45.649061 28007 master.cc:537] Master@127.20.93.189:38591 shutting down...
I20260810 11:44:45.648782 28053 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 11:44:45.650480 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:41177
--webserver_interface=127.20.93.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.188.keytab
--principal=oryx/127.20.93.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:44:45.652660 28112 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:44:45.657330 27489 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 11:44:45.666256 28007 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:45.666730 28007 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:45.666951 28007 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: stopping tablet replica
W20260810 11:44:45.667416 27489 consensus_peers.cc:487] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca -> Peer 0a8032ecd65c4c55b0ee3609108078e4 (127.20.93.189:38591): Couldn't send request to peer 0a8032ecd65c4c55b0ee3609108078e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 11:44:45.684803 28007 master.cc:552] Master@127.20.93.189:38591 shutdown complete.
I20260810 11:44:45.684947 28007 master_runner.cc:305] Clearing existing system tablet
I20260810 11:44:45.685446 28007 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:45.685545 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:45.688663 28007 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 11:44:45.690379 28119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:45.690724 28007 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:45.691008 28007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 11:44:45.691097 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "0a8032ecd65c4c55b0ee3609108078e4"
format_stamp: "Formatted at 2026-08-10 11:44:45 on dist-test-slave-njxd"
server_key: "e95c5adcfba348ca147ed27f82014055"
server_key_iv: "a374420dd73320c8886573dd3049a6b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:45.696322 28007 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:45.703315 28007 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 11:44:45.703505 28007 log.cc:1196] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 11:44:45.703943 28007 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Deleting consensus metadata
I20260810 11:44:45.704232 28007 master_runner.cc:315] Copying system tablet from 127.20.93.190:34189
I20260810 11:44:45.706909 28007 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: tablet copy: Beginning tablet copy session from remote peer at address 127.20.93.190:34189
I20260810 11:44:45.725286 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.707775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.20.93.190:34189)
0810 11:44:45.707970 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:45.707977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:45.710192 (+ 2215us) server_negotiation.cc:407] Connection header received
0810 11:44:45.710417 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:45.710424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:45.710515 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:45.710639 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:45.711766 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.712948 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.715033 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.715699 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.715865 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.718388 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:45.718395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:45.718401 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:45.718443 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:45.722119 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.722972 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.722978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.722982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.723136 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:45.723608 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:45.723614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:45.723617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:45.723997 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:45.724155 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:45.724663 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 11:44:45.724896 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:44:45.725315 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:45.707713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53344)
0810 11:44:45.710041 (+ 2328us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:45.710062 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:45.710251 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:45.710687 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:45.710692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:45.711298 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:45.711570 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.711582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.713105 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.713115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.714839 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:45.714854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:45.715878 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:45.715885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:45.716080 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:45.716885 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:45.716915 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:45.718133 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:45.722281 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.722287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.722292 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.722827 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.723268 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:45.723273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:45.723276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:45.723479 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:45.724140 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:45.724145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:45.724352 (+ 207us) client_negotiation.cc:769] Sending connection context
0810 11:44:45.724521 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 11:44:45.724844 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2163,"thread_start_us":2027,"threads_started":1}
I20260810 11:44:45.726693 27527 tablet_copy_service.cc:140] P 144d9109aa714068a117b47cf32e9fca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0a8032ecd65c4c55b0ee3609108078e4 ({username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:53344)
I20260810 11:44:45.726970 27527 tablet_copy_service.cc:161] P 144d9109aa714068a117b47cf32e9fca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0a8032ecd65c4c55b0ee3609108078e4 at {username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:53344: session id = 0a8032ecd65c4c55b0ee3609108078e4-00000000000000000000000000000000
I20260810 11:44:45.731411 27527 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 11:44:45.733888 28007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 11:44:45.740928 28007 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: tablet copy: Starting download of 0 data blocks...
I20260810 11:44:45.741251 28007 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: tablet copy: Starting download of 1 WAL segments...
I20260810 11:44:45.744347 28007 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 11:44:45.749185 27527 tablet_copy_service.cc:342] P 144d9109aa714068a117b47cf32e9fca: Request end of tablet copy session 0a8032ecd65c4c55b0ee3609108078e4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:53344
I20260810 11:44:45.749399 27527 tablet_copy_service.cc:433] P 144d9109aa714068a117b47cf32e9fca: ending tablet copy session 0a8032ecd65c4c55b0ee3609108078e4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0a8032ecd65c4c55b0ee3609108078e4
I20260810 11:44:45.762792 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:45.765749 28126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:45.766482 28128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:45.773488 28007 server_base.cc:1034] running on GCE node
W20260810 11:44:45.773422 28130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:45.774298 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:45.775118 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:45.776386 28007 hybrid_clock.cc:648] HybridClock initialized: now 1786362285776322 us; error 38 us; skew 500 ppm
I20260810 11:44:45.776760 28007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:45 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362285, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:45.779438 28007 init.cc:377] Logged in from keytab as oryx/127.20.93.189@KRBTEST.COM (short username oryx)
I20260810 11:44:45.786525 28007 webserver.cc:460] Webserver started at http://127.20.93.189:46097/ using document root <none> and password file <none>
I20260810 11:44:45.786996 28007 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:45.787156 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:45.790627 28007 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 11:44:45.792773 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:45.793264 28007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 11:44:45.793442 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "0a8032ecd65c4c55b0ee3609108078e4"
format_stamp: "Formatted at 2026-08-10 11:44:45 on dist-test-slave-njxd"
server_key: "e95c5adcfba348ca147ed27f82014055"
server_key_iv: "a374420dd73320c8886573dd3049a6b9"
server_key_version: "encryptionkey@0"
I20260810 11:44:45.793735 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:45.809270 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:45.813979 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:45.846453 28007 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.189:38591
I20260810 11:44:45.846453 28194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.189:38591 every 8 connection(s)
I20260810 11:44:45.847725 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
I20260810 11:44:45.855041 28195 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:44:45.857717 28195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Bootstrap starting.
I20260810 11:44:45.872373 28195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:44:45.872872 28195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Bootstrap complete.
I20260810 11:44:45.873847 28195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } }
I20260810 11:44:45.874087 28195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a8032ecd65c4c55b0ee3609108078e4, State: Initialized, Role: LEARNER
I20260810 11:44:45.874385 28195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } }
I20260810 11:44:45.875255 28199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } }
I20260810 11:44:45.875609 28195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:45.875684 28199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:44:45.891932 28211 catalog_manager.cc:1259] Loaded cluster ID: 04def2b6653248a2b8f356722d9a7daa
I20260810 11:44:45.892050 28211 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: loading cluster ID for follower catalog manager: success
I20260810 11:44:45.894732 28211 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: acquiring CA information for follower catalog manager: success
I20260810 11:44:45.896888 28211 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:44:45.971459 28000 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
W20260810 11:44:46.119846 28117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:46.120311 28117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:46.120477 28117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:46.132774 28117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:46.132934 28117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:46.132999 28117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:46.133083 28117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:46.133147 28117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:46.145270 28117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.188.keytab
--principal=oryx/127.20.93.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.93.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:46.146955 28117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:46.148372 28117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:46.161341 28221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:46.161267 28217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:46.161290 28218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:46.167138 28117 server_base.cc:1034] running on GCE node
I20260810 11:44:46.168054 28117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:46.169538 28117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:46.170809 28117 hybrid_clock.cc:648] HybridClock initialized: now 1786362286170761 us; error 46 us; skew 500 ppm
I20260810 11:44:46.171216 28117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362286, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:46.174580 28117 init.cc:377] Logged in from keytab as oryx/127.20.93.188@KRBTEST.COM (short username oryx)
I20260810 11:44:46.176086 28117 webserver.cc:460] Webserver started at http://127.20.93.188:34419/ using document root <none> and password file <none>
I20260810 11:44:46.176973 28117 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:46.177126 28117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:46.177456 28117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:46.181119 28117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/instance:
uuid: "888c0f9f38b74bb0a073df3938343815"
format_stamp: "Formatted at 2026-08-10 11:44:46 on dist-test-slave-njxd"
server_key: "2da26f2329fe01f0e4b964b25eda30eb"
server_key_iv: "61e96177e041d6c942497168ef8eda37"
server_key_version: "encryptionkey@0"
I20260810 11:44:46.181988 28117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal/instance:
uuid: "888c0f9f38b74bb0a073df3938343815"
format_stamp: "Formatted at 2026-08-10 11:44:46 on dist-test-slave-njxd"
server_key: "2da26f2329fe01f0e4b964b25eda30eb"
server_key_iv: "61e96177e041d6c942497168ef8eda37"
server_key_version: "encryptionkey@0"
I20260810 11:44:46.187402 28117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:44:46.191038 28228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:46.192379 28117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:44:46.192531 28117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "888c0f9f38b74bb0a073df3938343815"
format_stamp: "Formatted at 2026-08-10 11:44:46 on dist-test-slave-njxd"
server_key: "2da26f2329fe01f0e4b964b25eda30eb"
server_key_iv: "61e96177e041d6c942497168ef8eda37"
server_key_version: "encryptionkey@0"
I20260810 11:44:46.192701 28117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:46.216059 28117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:46.217374 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.207922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.20.93.189:38591)
0810 11:44:46.209978 (+ 2056us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.209984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.210008 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:46.210083 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.210088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.210173 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.210302 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:46.210837 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.212341 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.214786 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.216230 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.216356 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.216607 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.216888 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.217028 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1916,"thread_start_us":1765,"threads_started":1}
I20260810 11:44:46.217425 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.207748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39366)
0810 11:44:46.208198 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.208232 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.208379 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.210386 (+ 2007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.210392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.210435 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:46.210656 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.210666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.212517 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.212523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.214595 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.214612 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.216448 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.216456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.216654 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.216729 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.216847 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.217019 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":237,"threads_started":1}
I20260810 11:44:46.219051 28168 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 11:44:46.219724 28199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144d9109aa714068a117b47cf32e9fca. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } }
I20260810 11:44:46.220021 28199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:44:46.221828 28117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:46.222108 28117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:46.241734 28117 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.188:41177
I20260810 11:44:46.241744 28284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.188:41177 every 8 connection(s)
I20260810 11:44:46.243163 28117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
I20260810 11:44:46.247171 28285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:46.248543 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 28117
I20260810 11:44:46.256964 28285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362286, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.188@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:46.273290 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.250824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.20.93.188:41177)
0810 11:44:46.251692 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.251709 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.251763 (+ 54us) server_negotiation.cc:407] Connection header received
0810 11:44:46.253675 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.253706 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.254282 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.254791 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.256122 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.258254 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.259649 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.260591 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.260811 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.265830 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.265837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.265847 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.265881 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.269044 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.269776 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.269788 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.269795 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.269964 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.270392 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.270398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.270401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.270994 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.271317 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.271809 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.272226 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":423,"threads_started":1}
I20260810 11:44:46.275260 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.260194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.20.93.190:34189)
0810 11:44:46.260794 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.260800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.261368 (+ 568us) server_negotiation.cc:407] Connection header received
0810 11:44:46.261530 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.261535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.261642 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.261768 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.262873 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.263623 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.265367 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.265852 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.265983 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.269360 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.269365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.269368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.269399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.272390 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.273121 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.273127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.273131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.273262 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.273692 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.273697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.273701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.273975 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.274129 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.274641 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.274869 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":289,"threads_started":1}
I20260810 11:44:46.275308 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.260202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53360)
0810 11:44:46.261184 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.261238 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.261411 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.261852 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.261860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.262482 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.262721 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.262729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.263758 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.263765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.265197 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.265210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.266037 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.266044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.266243 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.266894 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.266910 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.269161 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.272523 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.272535 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.272542 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.272997 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.273385 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.273390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.273393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.273568 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.274117 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.274125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.274307 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.274536 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.274866 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":548,"threads_started":1}
I20260810 11:44:46.276772 28285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362286, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.188@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
I20260810 11:44:46.289799 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.277532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39372)
0810 11:44:46.277739 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.277759 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.277848 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.278129 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.278132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.278445 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.278584 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.278591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.279721 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.279724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.280756 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.280767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.281554 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.281559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.281747 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.282339 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.282356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.284303 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.287556 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.287567 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.287569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.287940 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.288320 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.288323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.288325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.288454 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.288907 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.288910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.289048 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.289226 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.289470 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 11:44:46.289911 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.277675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.20.93.189:38591)
0810 11:44:46.277868 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.277872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.277888 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:44:46.277937 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.277941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.277998 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.278077 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.278715 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.279595 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.280884 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.281372 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.281542 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.284537 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.284542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.284546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.284576 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.287426 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.288039 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.288043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.288045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.288223 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.288545 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.288548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.288550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.288811 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.288920 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.289318 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.289496 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:44:46.291332 28285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362286, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.188@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
I20260810 11:44:46.303467 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.292047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.0.0.1:41488)
0810 11:44:46.292241 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.292255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.292357 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.292677 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.292681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.293037 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.293172 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.293180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.294185 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.294192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.295043 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.295054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.295737 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.295742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.295974 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.296570 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.296598 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.298628 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.301399 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.301403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.301405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.301818 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.302113 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.302116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.302118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.302263 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.302740 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.302743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.302883 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.302990 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.303166 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":42}
I20260810 11:44:46.303541 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.292291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41488 (local address 127.20.93.188:41177)
0810 11:44:46.292459 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.292464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.292485 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:44:46.292517 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.292520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.292566 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.292662 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.293302 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.294065 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.295176 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.295580 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.295722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.298777 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.298781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.298784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.298811 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.301289 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.301912 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.301915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.301917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.302020 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.302352 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.302355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.302357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.302637 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.302740 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.303076 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.303268 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":58}
I20260810 11:44:46.308331 28285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Bootstrap starting.
I20260810 11:44:46.311596 28285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:46.312615 28285 log.cc:824] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:46.315239 28285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: No bootstrap required, opened a new log
I20260810 11:44:46.322412 28285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } }
I20260810 11:44:46.322746 28285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:46.322868 28285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888c0f9f38b74bb0a073df3938343815, State: Initialized, Role: FOLLOWER
I20260810 11:44:46.323421 28285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } }
I20260810 11:44:46.324283 28294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } } }
I20260810 11:44:46.324612 28294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:46.325277 28285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:44:46.334821 28307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:44:46.334975 28307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:44:46.338986 28117 master_runner.cc:428] Detected that this master 888c0f9f38b74bb0a073df3938343815 is joining an existing cluster
I20260810 11:44:46.339115 28117 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.93.188:41177
I20260810 11:44:46.364287 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.343658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.0.0.1:41492)
0810 11:44:46.344645 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.344666 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.345002 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.345626 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.345634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.346627 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.347098 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.347114 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.349078 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.349086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.351285 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.351305 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.352704 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.352713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.352908 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.353744 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.353775 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.356162 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.360850 (+ 4688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.360855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.360858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.361673 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.362136 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.362142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.362147 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.362335 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.363085 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.363090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.363265 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.363404 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.363705 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"spinlock_wait_cycles":24832,"thread_start_us":698,"threads_started":1}
I20260810 11:44:46.365432 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.343905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.20.93.188:41177)
0810 11:44:46.344112 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.344120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.345129 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 11:44:46.345191 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.345199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.345289 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.345394 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.347300 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.348825 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.351483 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.352279 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.352390 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.356338 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.356345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.356350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.356396 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.360710 (+ 4314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.361805 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.361813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.361818 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.361994 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.362446 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.362450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.362455 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.362885 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.363036 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.364851 (+ 1815us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.365062 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:44:46.369418 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.343444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.20.93.190:34189)
0810 11:44:46.343650 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.343657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.351849 (+ 8192us) server_negotiation.cc:407] Connection header received
0810 11:44:46.352381 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.352387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.352485 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.352562 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.354243 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.355438 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.357619 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.358341 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.358447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.362588 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.362595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.362600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.362642 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.366132 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.366974 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.366980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.366983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.367147 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.367686 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.367694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.367698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.368083 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.368261 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.368806 (+ 545us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.369060 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:44:46.370327 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.344037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53374)
0810 11:44:46.351593 (+ 7556us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.351614 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.352194 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.352781 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.352786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.353678 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.354028 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.354042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.355654 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.355659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.357417 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.357434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.358914 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.358922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.359330 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.360838 (+ 1508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.360861 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.362344 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.366333 (+ 3989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.366338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.366342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.366802 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.367310 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.367315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.367319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.367518 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.368240 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.368245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.368488 (+ 243us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.369518 (+ 1030us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.369872 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7399,"thread_start_us":15705,"threads_started":1}
I20260810 11:44:46.375231 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.343444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.20.93.189:38591)
0810 11:44:46.343649 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.343658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.360443 (+ 16785us) server_negotiation.cc:407] Connection header received
0810 11:44:46.360653 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.360659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.360766 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.360848 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.362474 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.363518 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.365525 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.366196 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.366308 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.368741 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.368746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.368750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.368782 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.372054 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.372863 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.372869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.372872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.373038 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.373556 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.373562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.373566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.374009 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.374164 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.374636 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.374878 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:44:46.375296 28314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.344939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39386)
0810 11:44:46.360125 (+ 15186us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.360148 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.360462 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.361098 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.361103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.361878 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.362292 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.362303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.363694 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.363699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.365348 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.365361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.366596 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.366602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.366778 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.367421 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.367447 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.368554 (+ 1107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.372216 (+ 3662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.372221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.372226 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.372703 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.373203 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.373207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.373211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.373423 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.374126 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.374130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.374337 (+ 207us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.374526 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.374834 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15026,"thread_start_us":6936,"threads_started":1}
I20260810 11:44:46.379528 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27572
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362283, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.190@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
I20260810 11:44:46.391727 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.377283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.0.0.1:41504)
0810 11:44:46.377457 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.377477 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.377611 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.378052 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.378057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.378530 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.378714 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.378725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.380221 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.380226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.381422 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.381435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.382344 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.382349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.382533 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.383118 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.383136 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.385429 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.388879 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.388884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.388889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.389503 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.389983 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.389988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.389992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.390143 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.390783 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.390789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.390992 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.391116 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.391394 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 11:44:46.392712 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.377338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.20.93.188:41177)
0810 11:44:46.377506 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.377514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.377539 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:44:46.377789 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.377794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.377898 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.378010 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.378894 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.380053 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.381670 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.382175 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.382330 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.385755 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.385760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.385763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.385793 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.388741 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.389676 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.389682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.389685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.389849 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.390277 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.390282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.390286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.390631 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.391979 (+ 1348us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.392125 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.392367 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:44:46.394205 27505 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.93.188:41177: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "888c0f9f38b74bb0a073df3938343815"
member_type: NON_VOTER
last_known_addr {
host: "127.20.93.188"
port: 41177
}
attrs {
promote: true
}
}
dest_uuid: "144d9109aa714068a117b47cf32e9fca"
cas_config_opid_index: 5
I20260810 11:44:46.395201 27505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } }
I20260810 11:44:46.397346 28233 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 888c0f9f38b74bb0a073df3938343815 (127.20.93.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.402524 28168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 11:44:46.403445 28259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:46.408751 28259 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.129@KRBTEST.COM
I20260810 11:44:46.410604 28233 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.93.188:41177
I20260810 11:44:46.411458 28117 master.cc:537] Master@127.20.93.188:41177 shutting down...
I20260810 11:44:46.413605 28320 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 11:44:46.426585 28318 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.003s
I20260810 11:44:46.429102 28320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.429446 28320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:46.434877 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.380638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.20.93.129:44303)
0810 11:44:46.396942 (+ 16304us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.396952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.396984 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:44:46.397060 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.397069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.397692 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.399935 (+ 2243us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.400206 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.401899 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.403901 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.404827 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.409153 (+ 4326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.411627 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.411649 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.411674 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.413426 (+ 1752us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.423024 (+ 9598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.425338 (+ 2314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.425347 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.425351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.425633 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.428299 (+ 2666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.428303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.428306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.428899 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.433514 (+ 4615us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.433775 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.434255 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16105,"thread_start_us":16595,"threads_started":1}
I20260810 11:44:46.435813 28168 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 888c0f9f38b74bb0a073df3938343815 (127.20.93.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.438187 27704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:46.447404 28199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: NON_VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: true } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.447755 28199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:44:46.448949 28320 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: attempting to promote NON_VOTER 0a8032ecd65c4c55b0ee3609108078e4 to VOTER
I20260810 11:44:46.449824 28320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } }
I20260810 11:44:46.452232 28168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 LEARNER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 11:44:46.452885 28319 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 11:44:46.455528 28318 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0a8032ecd65c4c55b0ee3609108078e4 (127.20.93.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.456408 28168 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0a8032ecd65c4c55b0ee3609108078e4 (127.20.93.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.460152 28320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.460115 28319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.460433 28320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:46.460461 28319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:46.461526 28199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.461805 28199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:46.464564 28117 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 11:44:46.464927 27491 consensus_peers.cc:487] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca -> Peer 888c0f9f38b74bb0a073df3938343815 (127.20.93.188:41177): Couldn't send request to peer 888c0f9f38b74bb0a073df3938343815. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 11:44:46.465039 28117 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:46.465262 28117 tablet_replica.cc:330] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: stopping tablet replica
I20260810 11:44:46.471650 28117 master.cc:552] Master@127.20.93.188:41177 shutdown complete.
I20260810 11:44:46.471767 28117 master_runner.cc:305] Clearing existing system tablet
I20260810 11:44:46.472137 28117 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:46.472266 28117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:46.474639 28117 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 11:44:46.476233 28326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:46.476783 28117 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:46.477149 28117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 11:44:46.477293 28117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "888c0f9f38b74bb0a073df3938343815"
format_stamp: "Formatted at 2026-08-10 11:44:46 on dist-test-slave-njxd"
server_key: "2da26f2329fe01f0e4b964b25eda30eb"
server_key_iv: "61e96177e041d6c942497168ef8eda37"
server_key_version: "encryptionkey@0"
I20260810 11:44:46.481158 28117 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:46.487283 28117 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 11:44:46.487481 28117 log.cc:1196] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 11:44:46.487948 28117 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Deleting consensus metadata
I20260810 11:44:46.488224 28117 master_runner.cc:315] Copying system tablet from 127.20.93.190:34189
I20260810 11:44:46.495576 28117 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: tablet copy: Beginning tablet copy session from remote peer at address 127.20.93.190:34189
I20260810 11:44:46.509941 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.496631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.20.93.190:34189)
0810 11:44:46.496900 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.496906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.497449 (+ 543us) server_negotiation.cc:407] Connection header received
0810 11:44:46.497722 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.497728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.497827 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.497940 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:46.498829 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.499593 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.500949 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.501414 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.501551 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.503567 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:46.503573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:46.503584 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:46.503618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:46.507003 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.507740 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.507745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.507749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.507895 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:46.508311 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:46.508316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:46.508318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:46.508672 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:46.508848 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.509279 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.509504 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260810 11:44:46.509995 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.496472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53386)
0810 11:44:46.497318 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.497338 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.497535 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.497967 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.497971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.498452 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:46.498664 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.498673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.499813 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.499818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.500783 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.500795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.501735 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.501747 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.501886 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.502573 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:46.502596 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:46.503355 (+ 759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:46.507143 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.507148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.507152 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.507590 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.508014 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:46.508019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:46.508023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:46.508191 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:46.508805 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:46.508811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:46.509004 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.509160 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.509519 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":566,"threads_started":1}
I20260810 11:44:46.510996 27527 tablet_copy_service.cc:140] P 144d9109aa714068a117b47cf32e9fca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 888c0f9f38b74bb0a073df3938343815 ({username='oryx', principal='oryx/127.20.93.188@KRBTEST.COM'} at 127.0.0.1:53386)
I20260810 11:44:46.511195 27527 tablet_copy_service.cc:161] P 144d9109aa714068a117b47cf32e9fca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 888c0f9f38b74bb0a073df3938343815 at {username='oryx', principal='oryx/127.20.93.188@KRBTEST.COM'} at 127.0.0.1:53386: session id = 888c0f9f38b74bb0a073df3938343815-00000000000000000000000000000000
I20260810 11:44:46.513689 27527 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 11:44:46.515457 28117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 11:44:46.520292 28117 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: tablet copy: Starting download of 0 data blocks...
I20260810 11:44:46.520596 28117 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: tablet copy: Starting download of 1 WAL segments...
I20260810 11:44:46.523015 28117 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 11:44:46.526573 27527 tablet_copy_service.cc:342] P 144d9109aa714068a117b47cf32e9fca: Request end of tablet copy session 888c0f9f38b74bb0a073df3938343815-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.93.188@KRBTEST.COM'} at 127.0.0.1:53386
I20260810 11:44:46.526749 27527 tablet_copy_service.cc:433] P 144d9109aa714068a117b47cf32e9fca: ending tablet copy session 888c0f9f38b74bb0a073df3938343815-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 888c0f9f38b74bb0a073df3938343815
I20260810 11:44:46.537845 28117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:46.540601 28333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:46.543041 28334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:46.554960 28336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:46.555490 28117 server_base.cc:1034] running on GCE node
I20260810 11:44:46.556102 28117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:46.556802 28117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:46.558121 28117 hybrid_clock.cc:648] HybridClock initialized: now 1786362286558072 us; error 29 us; skew 500 ppm
I20260810 11:44:46.558409 28117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:46 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362286, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:46.560468 28117 init.cc:377] Logged in from keytab as oryx/127.20.93.188@KRBTEST.COM (short username oryx)
I20260810 11:44:46.562032 28117 webserver.cc:460] Webserver started at http://127.20.93.188:41375/ using document root <none> and password file <none>
I20260810 11:44:46.562363 28117 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:46.562480 28117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:46.564991 28117 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 11:44:46.566820 28343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:46.567559 28117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 11:44:46.567698 28117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "888c0f9f38b74bb0a073df3938343815"
format_stamp: "Formatted at 2026-08-10 11:44:46 on dist-test-slave-njxd"
server_key: "2da26f2329fe01f0e4b964b25eda30eb"
server_key_iv: "61e96177e041d6c942497168ef8eda37"
server_key_version: "encryptionkey@0"
I20260810 11:44:46.567888 28117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:46.584329 28117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:46.589114 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27572
I20260810 11:44:46.590030 28117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:46.617035 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:44303
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:34189
--webserver_port=41429
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:44:46.631598 28117 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.188:41177
I20260810 11:44:46.631613 28399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.188:41177 every 8 connection(s)
I20260810 11:44:46.632727 28117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
I20260810 11:44:46.640561 28400 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:44:46.646340 28400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Bootstrap starting.
I20260810 11:44:46.667603 28400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:44:46.668315 28400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: Bootstrap complete.
I20260810 11:44:46.669448 28400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } }
I20260810 11:44:46.669798 28400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 888c0f9f38b74bb0a073df3938343815, State: Initialized, Role: LEARNER
I20260810 11:44:46.670061 28400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } }
I20260810 11:44:46.674204 28403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.674495 28400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:46.674777 28403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:44:46.689484 28416 catalog_manager.cc:1259] Loaded cluster ID: 04def2b6653248a2b8f356722d9a7daa
I20260810 11:44:46.689630 28416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: loading cluster ID for follower catalog manager: success
I20260810 11:44:46.692238 28416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: acquiring CA information for follower catalog manager: success
I20260810 11:44:46.694181 28416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:44:46.835726 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.826957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.0.0.1:41514)
0810 11:44:46.827267 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:46.827291 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:46.827461 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:46.828539 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:46.828544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:46.828579 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:46.828818 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.828833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.830960 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.830966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.832943 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:46.832960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.834824 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.834831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:46.835066 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.835115 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:44:46.835199 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 11:44:46.835373 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260810 11:44:46.837039 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:46.827248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.20.93.188:41177)
0810 11:44:46.828113 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:46.828120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:46.828142 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:44:46.828221 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:46.828230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:46.828339 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:46.828474 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:46.829000 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.830708 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.833122 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:46.834589 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:46.836160 (+ 1571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:46.836443 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:46.836505 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 11:44:46.836658 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":499,"threads_started":1}
I20260810 11:44:46.838212 28371 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 11:44:46.838900 28403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144d9109aa714068a117b47cf32e9fca. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:46.839257 28403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: LEARNER
W20260810 11:44:47.018671 28398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:47.018960 28398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:47.019151 28398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:47.027997 28398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:47.028131 28398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:47.028328 28398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:47.039589 28398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:44303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=41429
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:47.041062 28398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:47.042619 28398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:47.056753 28423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:47.056800 28422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:47.056872 28425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:47.058254 28398 server_base.cc:1034] running on GCE node
I20260810 11:44:47.058951 28398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:47.060097 28398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:47.061331 28398 hybrid_clock.cc:648] HybridClock initialized: now 1786362287061279 us; error 40 us; skew 500 ppm
I20260810 11:44:47.061784 28398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:47.065399 28398 init.cc:377] Logged in from keytab as oryx/127.20.93.129@KRBTEST.COM (short username oryx)
I20260810 11:44:47.067312 28398 webserver.cc:460] Webserver started at http://127.20.93.129:41429/ using document root <none> and password file <none>
I20260810 11:44:47.068140 28398 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:47.068298 28398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:47.076745 28398 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 11:44:47.080298 28433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.082089 28398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 11:44:47.082288 28398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "4dc17f0a5c6043b8a6131b6641f19ef0"
format_stamp: "Formatted at 2026-08-10 11:44:43 on dist-test-slave-njxd"
server_key: "1e6ec87f7fe1096466edf8ee555c3437"
server_key_iv: "d3e3b340a98f48cac36d98702dab774e"
server_key_version: "encryptionkey@0"
I20260810 11:44:47.083370 28398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:47.114009 28398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:47.119064 28398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:47.119402 28398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:47.120610 28398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:47.122192 28398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:47.122321 28398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.122463 28398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:47.122604 28398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.160495 28398 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:44303
I20260810 11:44:47.160562 28553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:44303 every 8 connection(s)
I20260810 11:44:47.162210 28398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:47.172091 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 28398
I20260810 11:44:47.172500 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27717
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.130@KRBTEST.COM
I20260810 11:44:47.193776 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.165327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:42141 (local address 127.20.93.188:41177)
0810 11:44:47.165612 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.165620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.169547 (+ 3927us) server_negotiation.cc:407] Connection header received
0810 11:44:47.170545 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.170551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.170655 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.170776 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.173537 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.174779 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.177838 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.178539 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.178652 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.185471 (+ 6819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.185478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.185483 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.185529 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.189904 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191129 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.191135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.191139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.191298 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191915 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.191922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.191927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.192311 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.192459 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.193128 (+ 669us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.193378 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:44:47.193851 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.165268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:57699 (local address 127.20.93.190:34189)
0810 11:44:47.165825 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.165832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.168856 (+ 3024us) server_negotiation.cc:407] Connection header received
0810 11:44:47.170584 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.170591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.170683 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.170774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.173713 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.174888 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.177822 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.178720 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.178855 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.184624 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.184629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.184633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.184672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.189238 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.190527 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.190533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.190538 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.190742 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191405 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.191412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.191418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.191818 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.191965 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.193137 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.193410 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":309,"threads_started":1}
I20260810 11:44:47.193812 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.165313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:33237 (local address 127.20.93.189:38591)
0810 11:44:47.165993 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.165999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.168875 (+ 2876us) server_negotiation.cc:407] Connection header received
0810 11:44:47.170570 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.170577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.170683 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.170774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.175506 (+ 4732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.176673 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.178703 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.179370 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.179491 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.185779 (+ 6288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.185785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.185790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.185856 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.189986 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191129 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.191135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.191139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.191299 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191982 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.191988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.191991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.192325 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.192459 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.193128 (+ 669us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.193375 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":1284,"threads_started":1}
I20260810 11:44:47.196184 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.166580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:57699)
0810 11:44:47.167596 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.167653 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.170239 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.171060 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.171070 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.172203 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.173350 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.173375 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.175170 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.175178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.177624 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.177646 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.179056 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.179065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.179530 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.180959 (+ 1429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.180980 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.184281 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.189514 (+ 5233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.189526 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.189547 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.190349 (+ 802us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.190956 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.190962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.190968 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.191267 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.192076 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.192086 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.192630 (+ 544us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.194154 (+ 1524us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.194770 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":796,"spinlock_wait_cycles":7168,"thread_start_us":462,"threads_started":1}
I20260810 11:44:47.196239 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.166596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.20.93.129:42141)
0810 11:44:47.169426 (+ 2830us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.169442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.170249 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.171059 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.171068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.172207 (+ 1139us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.173348 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.173370 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.174952 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.174958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.177646 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.177663 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.179055 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.179062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.179513 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.180533 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.180581 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.185123 (+ 4542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.190106 (+ 4983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.190114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.190117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.190939 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.191452 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.191461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.191767 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.192552 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.192556 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.192792 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.195336 (+ 2544us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.195627 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2688,"mutex_wait_us":428,"spinlock_wait_cycles":69376,"thread_start_us":2336,"threads_started":1}
I20260810 11:44:47.196774 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.166580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.20.93.129:33237)
0810 11:44:47.168761 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.168779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.170239 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.171066 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.171070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.172269 (+ 1199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.175294 (+ 3025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.175307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.176904 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.176910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.178525 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.178538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.179883 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.179890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.180094 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.180654 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.180676 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.185465 (+ 4789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.190131 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.190135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.190138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.190939 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.191452 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.191457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.191461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.191767 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.192553 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.192557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.192792 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.194700 (+ 1908us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.195143 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2009,"mutex_wait_us":2216,"spinlock_wait_cycles":80128,"thread_start_us":1648,"threads_started":1}
I20260810 11:44:47.198289 28554 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:47.198673 28554 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.198768 28556 heartbeater.cc:347] Connected to a master server at 127.20.93.189:38591
I20260810 11:44:47.198885 28556 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.199636 28555 heartbeater.cc:347] Connected to a master server at 127.20.93.188:41177
I20260810 11:44:47.199798 28555 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.200119 28554 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:47.200117 28556 heartbeater.cc:511] Master 127.20.93.189:38591 requested a full tablet report, sending...
I20260810 11:44:47.200474 28555 heartbeater.cc:511] Master 127.20.93.188:41177 requested a full tablet report, sending...
I20260810 11:44:47.202219 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.174298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.20.93.130:44411)
0810 11:44:47.175910 (+ 1612us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.175918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.175974 (+ 56us) server_negotiation.cc:407] Connection header received
0810 11:44:47.177905 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.177915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.178212 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.178627 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.181664 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.183265 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.185250 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.186210 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.186367 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.193103 (+ 6736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.193123 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.193143 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.193189 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.198164 (+ 4975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.199250 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.199261 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.199266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.199448 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.200001 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.200008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.200027 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.200567 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.200874 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.201500 (+ 626us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.201828 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":1355,"threads_started":1}
I20260810 11:44:47.202941 28156 ts_manager.cc:194] Registered new tserver with Master: 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303)
I20260810 11:44:47.203652 27849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:47.204128 27506 ts_manager.cc:194] Re-registered known tserver with Master: 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303)
I20260810 11:44:47.206063 27506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:57699
I20260810 11:44:47.207497 28361 ts_manager.cc:194] Registered new tserver with Master: 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303)
I20260810 11:44:47.300189 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27717
I20260810 11:44:47.320991 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:44411
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:34189
--webserver_port=36415
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:44:47.430924 28567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: NON_VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: true } } }
I20260810 11:44:47.431761 28567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:47.432827 28569 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: attempting to promote NON_VOTER 888c0f9f38b74bb0a073df3938343815 to VOTER
I20260810 11:44:47.433669 28569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } }
I20260810 11:44:47.436056 28371 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 LEARNER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 11:44:47.436190 28168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Refusing update from remote peer 144d9109aa714068a117b47cf32e9fca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 11:44:47.436914 28569 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 11:44:47.437448 28568 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 11:44:47.442214 28569 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 888c0f9f38b74bb0a073df3938343815 (127.20.93.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.443390 28168 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 888c0f9f38b74bb0a073df3938343815 (127.20.93.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.445717 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.446089 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:47.445972 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.446261 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:47.446532 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.446167 28371 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 888c0f9f38b74bb0a073df3938343815 (127.20.93.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.446800 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:47.450155 28572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.450625 28572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:47.451874 28573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "144d9109aa714068a117b47cf32e9fca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:47.452260 28573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 11:44:47.709411 28566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:47.709848 28566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:47.710026 28566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:47.717829 28566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:47.718009 28566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:47.718190 28566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:47.729482 28566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:44411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=36415
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:47.731069 28566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:47.732613 28566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:47.747328 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:47.747386 28582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:47.747434 28579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:47.747786 28566 server_base.cc:1034] running on GCE node
I20260810 11:44:47.748723 28566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:47.749969 28566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:47.751255 28566 hybrid_clock.cc:648] HybridClock initialized: now 1786362287751186 us; error 50 us; skew 500 ppm
I20260810 11:44:47.751644 28566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:47.755416 28566 init.cc:377] Logged in from keytab as oryx/127.20.93.130@KRBTEST.COM (short username oryx)
I20260810 11:44:47.756996 28566 webserver.cc:460] Webserver started at http://127.20.93.130:36415/ using document root <none> and password file <none>
I20260810 11:44:47.757779 28566 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:47.757910 28566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:47.764225 28566 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:44:47.767366 28592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.769120 28566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:44:47.769325 28566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "3408f93054024ae38fdb58991acd437b"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "5d7c8af60928f5d2039b605fcb7302cd"
server_key_iv: "019c7d023211321c14fc2d1aa6e35b89"
server_key_version: "encryptionkey@0"
I20260810 11:44:47.770588 28566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:47.785120 28566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:47.790171 28566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:47.790442 28566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:47.791584 28566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:47.793172 28566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:47.793320 28566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.793455 28566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:47.793545 28566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:47.829890 28566 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44411
I20260810 11:44:47.829986 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44411 every 8 connection(s)
I20260810 11:44:47.831547 28566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:47.842212 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 28566
I20260810 11:44:47.842626 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27864
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.131@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
Aug 10 11:44:47 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362287, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
I20260810 11:44:47.897922 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.844128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.20.93.131:39623)
0810 11:44:47.869571 (+ 25443us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.869607 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.869652 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:44:47.869770 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.869784 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.870163 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.873231 (+ 3068us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.873524 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.877198 (+ 3674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.879411 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.880341 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.880491 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.888099 (+ 7608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.888120 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.888142 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.888195 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.893094 (+ 4899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.895061 (+ 1967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.895073 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.895077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.895322 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.895820 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.895825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.895829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.896333 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.896659 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.897102 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.897413 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25177,"thread_start_us":26219,"threads_started":1}
I20260810 11:44:47.899644 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.834726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:41667 (local address 127.20.93.189:38591)
0810 11:44:47.878246 (+ 43520us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.878253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.878273 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:44:47.878340 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.878346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.878439 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.879830 (+ 1391us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.880604 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.881818 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.884935 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.885653 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.885775 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.891323 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.891331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.891336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.891396 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.895199 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.896286 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.896291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.896293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.896447 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.897019 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.897026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.897032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.897504 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.897679 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.898996 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.899248 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43367,"thread_start_us":41502,"threads_started":1}
I20260810 11:44:47.899767 27994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:47.901335 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.835592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.20.93.130:41667)
0810 11:44:47.844877 (+ 9285us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.844937 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.868166 (+ 23229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.878860 (+ 10694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.878865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.879570 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.880160 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.880188 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.882042 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.882048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.884752 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.884768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.886078 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.886088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.886439 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.887888 (+ 1449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.887912 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.891017 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.895386 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.895399 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.895419 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.896128 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.896589 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.896594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.896597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.896876 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.897957 (+ 1081us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.897966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.898482 (+ 516us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.898865 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.899467 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8908,"mutex_wait_us":59,"thread_start_us":39903,"threads_started":1}
I20260810 11:44:47.902405 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.836122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:42695)
0810 11:44:47.858227 (+ 22105us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.858256 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.863509 (+ 5253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.877626 (+ 14117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.877641 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.878777 (+ 1136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.880162 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.880188 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.883529 (+ 3341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.883534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.885114 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.885126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.886432 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.886438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.886818 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.887532 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.887573 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.891533 (+ 3960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.896987 (+ 5454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.896992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.896996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.897515 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.900154 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.900158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.900160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.900302 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.901233 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.901240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.901455 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.901624 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.901957 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21916,"mutex_wait_us":306,"spinlock_wait_cycles":21888,"thread_start_us":8269,"threads_started":1}
I20260810 11:44:47.903620 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.834719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:42695 (local address 127.20.93.190:34189)
0810 11:44:47.876686 (+ 41967us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.876692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.876711 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:44:47.876772 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.876777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.876860 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.877200 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.880789 (+ 3589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.883270 (+ 2481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.885332 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.886052 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.886173 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.891759 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.891768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.891776 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.891828 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.895804 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.897932 (+ 2128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.897937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.897941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.898113 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.900436 (+ 2323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.900442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.900445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.900858 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.901001 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.901935 (+ 934us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.902187 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41790,"thread_start_us":41066,"threads_started":1}
I20260810 11:44:47.903661 28716 heartbeater.cc:347] Connected to a master server at 127.20.93.189:38591
I20260810 11:44:47.904069 28716 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.904979 28716 heartbeater.cc:511] Master 127.20.93.189:38591 requested a full tablet report, sending...
I20260810 11:44:47.905870 28714 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:47.906021 28714 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.906540 28714 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:47.907078 28156 ts_manager.cc:194] Registered new tserver with Master: 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
I20260810 11:44:47.907684 27506 ts_manager.cc:194] Re-registered known tserver with Master: 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
I20260810 11:44:47.907977 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.834651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:60905 (local address 127.20.93.188:41177)
0810 11:44:47.876202 (+ 41551us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:47.876209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:47.876504 (+ 295us) server_negotiation.cc:407] Connection header received
0810 11:44:47.876739 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:47.876745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:47.876835 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:47.877245 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:47.880381 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.881641 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.884959 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.886131 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.886259 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.895164 (+ 8905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:47.895173 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:47.895181 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:47.895242 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:47.899557 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.900879 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.900885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.900891 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.901058 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:47.902968 (+ 1910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:47.902976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:47.902981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:47.903294 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:47.903460 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:47.907279 (+ 3819us) server_negotiation.cc:299] Negotiation successful
0810 11:44:47.907560 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41409,"thread_start_us":41265,"threads_started":1}
I20260810 11:44:47.908610 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:47.836921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.20.93.130:60905)
0810 11:44:47.876202 (+ 39281us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:47.876223 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:47.876565 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:47.877633 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:47.877642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:47.878774 (+ 1132us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:47.880164 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.880188 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.881883 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.881888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.884754 (+ 2866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:47.884773 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:47.886563 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:47.886568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:47.886856 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:47.887611 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:47.887639 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:47.894245 (+ 6606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:47.899731 (+ 5486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.899739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.899743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.900727 (+ 984us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.901187 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:47.901195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:47.901198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:47.901381 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:47.903444 (+ 2063us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:47.903449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:47.905912 (+ 2463us) client_negotiation.cc:769] Sending connection context
0810 11:44:47.906177 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 11:44:47.906658 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39159,"mutex_wait_us":307,"spinlock_wait_cycles":24576,"thread_start_us":20868,"threads_started":1}
I20260810 11:44:47.908986 27506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:42695
I20260810 11:44:47.910296 28715 heartbeater.cc:347] Connected to a master server at 127.20.93.188:41177
I20260810 11:44:47.910444 28715 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:47.911420 28715 heartbeater.cc:511] Master 127.20.93.188:41177 requested a full tablet report, sending...
I20260810 11:44:47.912639 28361 ts_manager.cc:194] Registered new tserver with Master: 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
I20260810 11:44:47.997368 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27864
I20260810 11:44:48.017834 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:39623
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:34189
--webserver_port=44899
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:44:48.210036 28554 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
W20260810 11:44:48.364794 28727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:48.365150 28727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:48.365330 28727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:48.373100 28727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:48.373284 28727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:48.373476 28727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:48.384462 28727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:39623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=44899
--tserver_master_addrs=127.20.93.190:34189,127.20.93.189:38591,127.20.93.188:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:48.385965 28727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:48.387436 28727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:48.401535 28733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:48.401724 28732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:48.401738 28735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:48.402952 28727 server_base.cc:1034] running on GCE node
I20260810 11:44:48.403709 28727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:48.404915 28727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:48.406142 28727 hybrid_clock.cc:648] HybridClock initialized: now 1786362288406067 us; error 69 us; skew 500 ppm
I20260810 11:44:48.406492 28727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:48 dist-test-slave-njxd krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362288, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:48.410033 28727 init.cc:377] Logged in from keytab as oryx/127.20.93.131@KRBTEST.COM (short username oryx)
I20260810 11:44:48.411621 28727 webserver.cc:460] Webserver started at http://127.20.93.131:44899/ using document root <none> and password file <none>
I20260810 11:44:48.412381 28727 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:48.412547 28727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:48.418530 28727 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:44:48.421511 28742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:48.422967 28727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 11:44:48.423166 28727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "13b0558dedac4a24bba11bcf72a51030"
format_stamp: "Formatted at 2026-08-10 11:44:44 on dist-test-slave-njxd"
server_key: "a8146dc480b5aed3e832d2fafd4e0146"
server_key_iv: "3f87eaaef70b7e3d3e342410d07f1a43"
server_key_version: "encryptionkey@0"
I20260810 11:44:48.424206 28727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:48.452023 28727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:48.456816 28727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:48.457134 28727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:48.458319 28727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:44:48.459817 28727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:48.459939 28727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:48.460075 28727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:48.460166 28727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:48.493644 28727 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:39623
I20260810 11:44:48.493921 28865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:39623 every 8 connection(s)
I20260810 11:44:48.495364 28727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:48.506017 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 28727
Aug 10 11:44:48 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362288, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
Aug 10 11:44:48 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362288, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.189@KRBTEST.COM
I20260810 11:44:48.554972 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.498315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:37913 (local address 127.20.93.190:34189)
0810 11:44:48.527616 (+ 29301us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.527625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.527654 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:44:48.527750 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.527759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.527876 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.528030 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.533743 (+ 5713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.534455 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.536534 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.537071 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.537205 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.544214 (+ 7009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.544222 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.544230 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.544289 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.548878 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.550119 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.550129 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.550135 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.550327 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.550985 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.550993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.550999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.551457 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.551642 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.554249 (+ 2607us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.554539 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29165,"thread_start_us":29044,"threads_started":1}
I20260810 11:44:48.557735 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.499115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:37913)
0810 11:44:48.521108 (+ 21993us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.521183 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.524750 (+ 3567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.529557 (+ 4807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.529589 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.532742 (+ 3153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:48.533151 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.533165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.534610 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.534614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.536407 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.536417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.537315 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.537323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.537695 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.538582 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:48.538615 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:48.543844 (+ 5229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:48.549086 (+ 5242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.549096 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.549116 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.549937 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.550488 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.550495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.550501 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.550834 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.551762 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:48.551770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:48.552330 (+ 560us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.555377 (+ 3047us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.556092 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21529,"mutex_wait_us":67,"thread_start_us":29266,"threads_started":1}
I20260810 11:44:48.559381 28866 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34189
I20260810 11:44:48.559697 28866 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:48.560585 28866 heartbeater.cc:511] Master 127.20.93.190:34189 requested a full tablet report, sending...
I20260810 11:44:48.562095 27506 ts_manager.cc:194] Re-registered known tserver with Master: 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:48.562984 27506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:37913
I20260810 11:44:48.569798 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.498314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35969 (local address 127.20.93.189:38591)
0810 11:44:48.527111 (+ 28797us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.527122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.529046 (+ 1924us) server_negotiation.cc:407] Connection header received
0810 11:44:48.529205 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.529210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.529310 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.529416 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.531391 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.532500 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.537538 (+ 5038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.538279 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.538426 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.556133 (+ 17707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.556141 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.556146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.556191 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.566306 (+ 10115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.567080 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.567089 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.567095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.567284 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.567810 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.567817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.567821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.568167 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.568304 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.568926 (+ 622us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.569174 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28596,"thread_start_us":28803,"threads_started":1}
Aug 10 11:44:48 dist-test-slave-njxd krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362288, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.188@KRBTEST.COM
I20260810 11:44:48.574869 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.499115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.20.93.131:35969)
0810 11:44:48.528864 (+ 29749us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.528889 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.529067 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.529627 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.529631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.530497 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:48.531191 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.531210 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.532697 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.532704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.536382 (+ 3678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.536405 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.538695 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.538704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.538912 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.539541 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:48.539557 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:48.554755 (+ 15198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:48.566519 (+ 11764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.566522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.566525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.566934 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.567421 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.567429 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.567434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.567646 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.568446 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:48.568453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:48.568600 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.573912 (+ 5312us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.574285 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29522,"mutex_wait_us":102,"spinlock_wait_cycles":70912,"thread_start_us":26056,"threads_started":1}
I20260810 11:44:48.580145 28868 heartbeater.cc:347] Connected to a master server at 127.20.93.189:38591
I20260810 11:44:48.580315 28868 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:48.580818 28868 heartbeater.cc:511] Master 127.20.93.189:38591 requested a full tablet report, sending...
I20260810 11:44:48.582075 28156 ts_manager.cc:194] Registered new tserver with Master: 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:48.584695 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.499116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.20.93.131:44125)
0810 11:44:48.535643 (+ 36527us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.535670 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.535830 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.538816 (+ 2986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.538824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.539877 (+ 1053us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:48.540162 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.540171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.567320 (+ 27149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.567329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.569178 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.569192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.571582 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.571590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.571801 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.572507 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:48.572531 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:48.576697 (+ 4166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:48.581165 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.581170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.581175 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.581710 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.582214 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:48.582222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:48.582226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:48.582413 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:48.583501 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:48.583507 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:48.583710 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.583862 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.584206 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36311,"mutex_wait_us":585,"thread_start_us":36190,"threads_started":1}
I20260810 11:44:48.586053 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.499474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:44125 (local address 127.20.93.188:41177)
0810 11:44:48.538075 (+ 38601us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.538086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.538113 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:44:48.538189 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.538197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.538304 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.538444 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.560313 (+ 21869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.567163 (+ 6850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.570457 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.571366 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.571529 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.576956 (+ 5427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.576963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.576968 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.577015 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.581020 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.581854 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.581861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.581864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.582009 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.582844 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.582849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.582853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.583251 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.583413 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.585262 (+ 1849us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.585623 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38418,"thread_start_us":39118,"threads_started":1}
I20260810 11:44:48.587211 28867 heartbeater.cc:347] Connected to a master server at 127.20.93.188:41177
I20260810 11:44:48.587390 28867 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:48.587780 28867 heartbeater.cc:511] Master 127.20.93.188:41177 requested a full tablet report, sending...
I20260810 11:44:48.589107 28361 ts_manager.cc:194] Registered new tserver with Master: 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:48.595093 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.564001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.20.93.188:41177)
0810 11:44:48.564576 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.564582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.575971 (+ 11389us) server_negotiation.cc:407] Connection header received
0810 11:44:48.577128 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.577134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.577219 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.577378 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.579906 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.581193 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.583511 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.584209 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.584317 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.588105 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.588112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.588116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.588151 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.591656 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.592461 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.592468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.592472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.592641 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.593094 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.593101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.593107 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.593440 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.593740 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.594422 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.594682 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":298,"threads_started":1}
I20260810 11:44:48.595636 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.541955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.20.93.189:38591)
0810 11:44:48.571040 (+ 29085us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.571049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.571069 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:44:48.571123 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.571131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.571213 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.571315 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.573921 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.575072 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.583053 (+ 7981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.583704 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.583831 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.588118 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.588123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.588126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.588155 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.591741 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.592747 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.592754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.592758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.592891 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.593328 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.593334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.593338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.593899 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.594209 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.594619 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.594864 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28951,"thread_start_us":33494,"threads_started":1}
I20260810 11:44:48.597874 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.541860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.20.93.190:34189)
0810 11:44:48.555298 (+ 13438us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.555308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.570245 (+ 14937us) server_negotiation.cc:407] Connection header received
0810 11:44:48.570385 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.570391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.570499 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.570614 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.576616 (+ 6002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.577892 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.581292 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.582045 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.582181 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.590409 (+ 8228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.590415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.590419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.590463 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.594074 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.595050 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.595056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.595060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.595237 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.596316 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.596321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.596324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.596726 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.596974 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.597239 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.597469 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13292,"thread_start_us":32709,"threads_started":1}
I20260810 11:44:48.601854 27506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:48.609038 27506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:48.652312 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.631004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.20.93.130:44411)
0810 11:44:48.634013 (+ 3009us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.634025 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.637886 (+ 3861us) server_negotiation.cc:407] Connection header received
0810 11:44:48.638032 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.638044 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.638404 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.638829 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:48.645751 (+ 6922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.647638 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.649371 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.650925 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.651057 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.651283 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.651829 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.652004 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2854,"thread_start_us":2756,"threads_started":1}
I20260810 11:44:48.653105 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.630986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.20.93.129:44303)
0810 11:44:48.631623 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.631637 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.634851 (+ 3214us) server_negotiation.cc:407] Connection header received
0810 11:44:48.635214 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.635223 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.635567 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.635923 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:48.639157 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.641293 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.649438 (+ 8145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.650937 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.651040 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.651281 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.652725 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.652895 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":265,"threads_started":1}
I20260810 11:44:48.654307 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.630422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44411 (local address 127.0.0.1:43156)
0810 11:44:48.637414 (+ 6992us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.637434 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.637731 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.644879 (+ 7148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.644887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.644938 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:48.645464 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.645479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.647830 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.647834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.649202 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.649215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.651417 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.651421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.651599 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.651646 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.651758 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.651903 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6819,"thread_start_us":6678,"threads_started":1}
I20260810 11:44:48.654410 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.630270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44303 (local address 127.0.0.1:40118)
0810 11:44:48.634557 (+ 4287us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.634578 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.634991 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.636283 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.636290 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.636326 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:48.636788 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.636803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.641549 (+ 4746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.641558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.644600 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.644623 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.652264 (+ 7641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.652270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.652468 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.652505 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.653429 (+ 924us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.653538 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4142,"thread_start_us":3961,"threads_started":1}
I20260810 11:44:48.658800 28641 tablet_service.cc:1467] Processing CreateTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b (DEFAULT_TABLE table=test-table [id=7216149b4ae84058a48c3c49ec93c74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:48.658800 28483 tablet_service.cc:1467] Processing CreateTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b (DEFAULT_TABLE table=test-table [id=7216149b4ae84058a48c3c49ec93c74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:48.660140 28483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23c0a8b799d487da52910bfcc2f0c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:48.660248 28641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23c0a8b799d487da52910bfcc2f0c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:48.664829 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.634738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:39623 (local address 127.0.0.1:35572)
0810 11:44:48.654074 (+ 19336us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.654094 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.655073 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.655974 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.655978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.656005 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:48.656396 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.656409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.659351 (+ 2942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.659358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.661487 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.661504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.663979 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.663985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.664162 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.664197 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.664319 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.664464 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19198,"thread_start_us":19042,"threads_started":1}
I20260810 11:44:48.664881 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.635097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.20.93.131:39623)
0810 11:44:48.647475 (+ 12378us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.647484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.654556 (+ 7072us) server_negotiation.cc:407] Connection header received
0810 11:44:48.655295 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.655305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.655584 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.656004 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:48.656631 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.659182 (+ 2551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.661692 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.663746 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.663889 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.664104 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.664375 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.664547 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12178,"thread_start_us":12397,"threads_started":1}
I20260810 11:44:48.669178 28796 tablet_service.cc:1467] Processing CreateTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b (DEFAULT_TABLE table=test-table [id=7216149b4ae84058a48c3c49ec93c74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:48.670794 28796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23c0a8b799d487da52910bfcc2f0c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:48.673975 28906 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Bootstrap starting.
I20260810 11:44:48.676659 28907 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Bootstrap starting.
I20260810 11:44:48.678601 28906 tablet_bootstrap.cc:654] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:48.679922 28906 log.cc:824] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:48.680145 28907 tablet_bootstrap.cc:654] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:48.681195 28907 log.cc:824] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:48.683281 28906 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: No bootstrap required, opened a new log
I20260810 11:44:48.683552 28906 ts_tablet_manager.cc:1397] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 11:44:48.684883 28907 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: No bootstrap required, opened a new log
I20260810 11:44:48.685223 28907 ts_tablet_manager.cc:1397] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 11:44:48.685909 28910 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Bootstrap starting.
I20260810 11:44:48.688963 28910 tablet_bootstrap.cc:654] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:48.690008 28910 log.cc:824] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:48.693001 28910 tablet_bootstrap.cc:492] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: No bootstrap required, opened a new log
I20260810 11:44:48.693262 28910 ts_tablet_manager.cc:1397] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260810 11:44:48.695292 28906 raft_consensus.cc:348] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.695751 28906 raft_consensus.cc:374] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:48.695870 28906 raft_consensus.cc:729] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc17f0a5c6043b8a6131b6641f19ef0, State: Initialized, Role: FOLLOWER
I20260810 11:44:48.696251 28907 raft_consensus.cc:348] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.696720 28907 raft_consensus.cc:374] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:48.696623 28906 consensus_queue.cc:260] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.696820 28907 raft_consensus.cc:729] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3408f93054024ae38fdb58991acd437b, State: Initialized, Role: FOLLOWER
I20260810 11:44:48.697538 28907 consensus_queue.cc:260] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.698799 28714 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
I20260810 11:44:48.699783 28907 ts_tablet_manager.cc:1428] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 11:44:48.702878 28910 raft_consensus.cc:348] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.703322 28910 raft_consensus.cc:374] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:48.703434 28910 raft_consensus.cc:729] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b0558dedac4a24bba11bcf72a51030, State: Initialized, Role: FOLLOWER
I20260810 11:44:48.704042 28910 consensus_queue.cc:260] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.704437 28906 ts_tablet_manager.cc:1428] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260810 11:44:48.707805 28866 heartbeater.cc:503] Master 127.20.93.190:34189 was elected leader, sending a full tablet report...
I20260810 11:44:48.708602 28910 ts_tablet_manager.cc:1428] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260810 11:44:48.736557 28912 raft_consensus.cc:482] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:48.736886 28912 raft_consensus.cc:504] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.739239 28912 leader_election.cc:288] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411), 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:48.746721 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.739824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:32885 (local address 127.20.93.131:39623)
0810 11:44:48.740038 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.740042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.741112 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 11:44:48.741337 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.741341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.741399 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.741512 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:48.742207 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.742967 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.744957 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.745765 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.745969 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.746051 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.746340 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.746471 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:44:48.746894 28916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.739831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:39623 (local address 127.20.93.129:32885)
0810 11:44:48.740883 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.740898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.741161 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.741527 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.741532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.741555 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:48.742034 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.742044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.743122 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.743129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.744799 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.744812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.745995 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.746002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.746169 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.746193 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.746302 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.746440 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":84,"thread_start_us":740,"threads_started":1}
I20260810 11:44:48.746968 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.739389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44411 (local address 127.20.93.129:41817)
0810 11:44:48.740707 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:48.740737 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:48.741124 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:48.741481 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:48.741486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:48.741510 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:48.741962 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.741972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.743269 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.743274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.744830 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:48.744842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.746110 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.746117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:48.746268 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.746291 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:44:48.746399 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:44:48.746530 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"mutex_wait_us":47,"thread_start_us":888,"threads_started":1}
I20260810 11:44:48.747231 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.739542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41817 (local address 127.20.93.130:44411)
0810 11:44:48.739789 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.739796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.741029 (+ 1233us) server_negotiation.cc:407] Connection header received
0810 11:44:48.741260 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.741266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.741345 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.741461 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:48.742132 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.743111 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.744992 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.745875 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.746059 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.746142 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.746426 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.746567 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:44:48.747911 28818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23c0a8b799d487da52910bfcc2f0c3b" candidate_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b0558dedac4a24bba11bcf72a51030" is_pre_election: true
I20260810 11:44:48.748090 28663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23c0a8b799d487da52910bfcc2f0c3b" candidate_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3408f93054024ae38fdb58991acd437b" is_pre_election: true
I20260810 11:44:48.748478 28818 raft_consensus.cc:2454] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dc17f0a5c6043b8a6131b6641f19ef0 in term 0.
I20260810 11:44:48.748624 28663 raft_consensus.cc:2454] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dc17f0a5c6043b8a6131b6641f19ef0 in term 0.
I20260810 11:44:48.749264 28435 leader_election.cc:302] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13b0558dedac4a24bba11bcf72a51030, 3408f93054024ae38fdb58991acd437b, 4dc17f0a5c6043b8a6131b6641f19ef0; no voters:
I20260810 11:44:48.749781 28912 raft_consensus.cc:2781] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:48.750007 28912 raft_consensus.cc:482] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:48.750124 28912 raft_consensus.cc:3037] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:48.752509 28912 raft_consensus.cc:504] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.753314 28912 leader_election.cc:288] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [CANDIDATE]: Term 1 election: Requested vote from peers 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411), 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623)
I20260810 11:44:48.753818 28663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23c0a8b799d487da52910bfcc2f0c3b" candidate_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3408f93054024ae38fdb58991acd437b"
I20260810 11:44:48.753816 28818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23c0a8b799d487da52910bfcc2f0c3b" candidate_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b0558dedac4a24bba11bcf72a51030"
I20260810 11:44:48.754033 28663 raft_consensus.cc:3037] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:48.754026 28818 raft_consensus.cc:3037] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:48.756294 28663 raft_consensus.cc:2454] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dc17f0a5c6043b8a6131b6641f19ef0 in term 1.
I20260810 11:44:48.756356 28818 raft_consensus.cc:2454] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dc17f0a5c6043b8a6131b6641f19ef0 in term 1.
I20260810 11:44:48.756810 28436 leader_election.cc:302] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3408f93054024ae38fdb58991acd437b, 4dc17f0a5c6043b8a6131b6641f19ef0; no voters:
I20260810 11:44:48.757117 28912 raft_consensus.cc:2781] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:48.757664 28912 raft_consensus.cc:686] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 LEADER]: Becoming Leader. State: Replica: 4dc17f0a5c6043b8a6131b6641f19ef0, State: Running, Role: LEADER
I20260810 11:44:48.758261 28912 consensus_queue.cc:237] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:48.766605 27503 catalog_manager.cc:5495] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:48.795217 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.789398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.20.93.129:44303)
0810 11:44:48.789668 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.789675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.789793 (+ 118us) server_negotiation.cc:407] Connection header received
0810 11:44:48.790049 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.790053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.790145 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.790342 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:48.790758 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.791445 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.792871 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.793288 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.793422 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.794055 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:48.794579 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:48.794705 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.794829 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.794954 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:44:48.803540 28818 raft_consensus.cc:1261] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Refusing update from remote peer 4dc17f0a5c6043b8a6131b6641f19ef0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:48.803543 28663 raft_consensus.cc:1261] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Refusing update from remote peer 4dc17f0a5c6043b8a6131b6641f19ef0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:48.804612 28917 consensus_queue.cc:1035] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:48.804984 28912 consensus_queue.cc:1035] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:48.819758 28921 mvcc.cc:205] Tried to move back new op lower bound from 7316939934929248256 to 7316939934758088704. Current Snapshot: MvccSnapshot[applied={T|T < 7316939934929248256}]
I20260810 11:44:48.823253 28920 mvcc.cc:205] Tried to move back new op lower bound from 7316939934929248256 to 7316939934758088704. Current Snapshot: MvccSnapshot[applied={T|T < 7316939934929248256 or (T in {7316939934929248256})}]
I20260810 11:44:48.841877 27506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:48.842195 27506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:48.856438 27506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 144d9109aa714068a117b47cf32e9fca: Sending DeleteTablet for 3 replicas of tablet f23c0a8b799d487da52910bfcc2f0c3b
I20260810 11:44:48.858811 28641 tablet_service.cc:1514] Processing DeleteTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:48 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43156
I20260810 11:44:48.860980 28483 tablet_service.cc:1514] Processing DeleteTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:48 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:40118
I20260810 11:44:48.864101 28796 tablet_service.cc:1514] Processing DeleteTablet for tablet f23c0a8b799d487da52910bfcc2f0c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:48 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:35572
I20260810 11:44:48.866158 28932 tablet_replica.cc:330] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: stopping tablet replica
I20260810 11:44:48.866669 28932 raft_consensus.cc:2229] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:48.867240 28932 raft_consensus.cc:2258] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:48.869699 28932 ts_tablet_manager.cc:1905] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:48.874413 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 27472
I20260810 11:44:48.876495 27537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:48.878536 28932 ts_tablet_manager.cc:1918] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:48.878743 28932 log.cc:1196] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/f23c0a8b799d487da52910bfcc2f0c3b
I20260810 11:44:48.879559 28932 ts_tablet_manager.cc:1939] T f23c0a8b799d487da52910bfcc2f0c3b P 3408f93054024ae38fdb58991acd437b: Deleting consensus metadata
I20260810 11:44:48.881863 27491 catalog_manager.cc:4841] TS 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411): tablet f23c0a8b799d487da52910bfcc2f0c3b (table test-table [id=7216149b4ae84058a48c3c49ec93c74b]) successfully deleted
I20260810 11:44:48.883529 28933 tablet_replica.cc:330] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: stopping tablet replica
I20260810 11:44:48.884037 28933 raft_consensus.cc:2229] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:48.884771 28933 raft_consensus.cc:2258] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:48.886157 28935 tablet_replica.cc:330] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: stopping tablet replica
I20260810 11:44:48.886878 28935 raft_consensus.cc:2229] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:48.887418 28935 raft_consensus.cc:2258] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:48.890125 28935 ts_tablet_manager.cc:1905] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:48.890232 28933 ts_tablet_manager.cc:1905] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:48.895561 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.863307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.20.93.189:38591)
0810 11:44:48.863495 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.863502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.868127 (+ 4625us) server_negotiation.cc:407] Connection header received
0810 11:44:48.868208 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.868223 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.868312 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.868419 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.871035 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.872328 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.874705 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.875395 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.875507 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.887342 (+ 11835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.887349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.887379 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.887434 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.891166 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.892351 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.892358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.892362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.892521 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.892977 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.892982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.892986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.893354 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.893674 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.894906 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.895152 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:44:48.898181 28933 ts_tablet_manager.cc:1918] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:48.898388 28933 log.cc:1196] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/f23c0a8b799d487da52910bfcc2f0c3b
I20260810 11:44:48.899060 28933 ts_tablet_manager.cc:1939] T f23c0a8b799d487da52910bfcc2f0c3b P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting consensus metadata
I20260810 11:44:48.905628 28935 ts_tablet_manager.cc:1918] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:48.905944 28935 log.cc:1196] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/f23c0a8b799d487da52910bfcc2f0c3b
I20260810 11:44:48.906730 28935 ts_tablet_manager.cc:1939] T f23c0a8b799d487da52910bfcc2f0c3b P 13b0558dedac4a24bba11bcf72a51030: Deleting consensus metadata
I20260810 11:44:48.917719 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:48.868023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.20.93.188:41177)
0810 11:44:48.868321 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:48.868328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:48.897617 (+ 29289us) server_negotiation.cc:407] Connection header received
0810 11:44:48.897894 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:48.897900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:48.898014 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:48.898086 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:48.902592 (+ 4506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.903785 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.906040 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:48.906727 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:48.906837 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:48.910733 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:48.910742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:48.910748 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:48.910801 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:48.914529 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.915426 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.915431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.915436 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.915605 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:48.916049 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:48.916055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:48.916058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:48.916410 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:48.916675 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:48.917082 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 11:44:48.917322 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 11:44:49.022316 27490 catalog_manager.cc:4841] TS 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623): tablet f23c0a8b799d487da52910bfcc2f0c3b (table test-table [id=7216149b4ae84058a48c3c49ec93c74b]) successfully deleted
I20260810 11:44:49.022676 27490 catalog_manager.cc:4841] TS 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303): tablet f23c0a8b799d487da52910bfcc2f0c3b (table test-table [id=7216149b4ae84058a48c3c49ec93c74b]) successfully deleted
I20260810 11:44:49.022989 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 27472
I20260810 11:44:49.096804 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:49.062122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.20.93.188:41177)
0810 11:44:49.062331 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:49.062338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:49.078858 (+ 16520us) server_negotiation.cc:407] Connection header received
0810 11:44:49.079692 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:49.079702 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:49.079835 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:49.079923 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:49.082617 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:49.083799 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:49.085908 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:49.086339 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:49.086487 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:49.090001 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:49.090007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:49.090010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:49.090045 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:49.093480 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:49.094393 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:49.094397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:49.094399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:49.094541 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:49.095005 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:49.095011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:49.095014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:49.095393 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:49.095704 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:49.096147 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 11:44:49.096406 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:44:49.097362 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:49.062231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.20.93.189:38591)
0810 11:44:49.062460 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:49.062467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:49.079737 (+ 17270us) server_negotiation.cc:407] Connection header received
0810 11:44:49.080579 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:49.080586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:49.080681 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:49.080759 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:49.083351 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:49.084352 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:49.086423 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:49.087124 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:49.087237 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:49.090844 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:49.090854 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:49.090862 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:49.090912 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:49.093877 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:49.094839 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:49.094845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:49.094849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:49.095014 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:49.095418 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:49.095424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:49.095429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:49.095854 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:49.096281 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:49.096802 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 11:44:49.097052 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 11:44:49.842195 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:49.841342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:57753)
0810 11:44:49.841906 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:49.842028 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":327,"threads_started":1}
W20260810 11:44:49.842748 28554 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:34189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:49.859550 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:49.858725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:58467)
0810 11:44:49.859203 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:49.859384 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":258,"threads_started":1}
W20260810 11:44:49.859990 28714 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:34189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:49.868525 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:49.867797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:58761)
0810 11:44:49.868221 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:49.868345 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":227,"threads_started":1}
W20260810 11:44:49.868963 28866 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:34189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.354981 28947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 144d9109aa714068a117b47cf32e9fca)
I20260810 11:44:50.355300 28947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } }
I20260810 11:44:50.356699 28947 leader_election.cc:288] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189), 0a8032ecd65c4c55b0ee3609108078e4 (127.20.93.189:38591)
I20260810 11:44:50.357887 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.356795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53798)
0810 11:44:50.357359 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.357627 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":556,"threads_started":1}
W20260810 11:44:50.358176 28348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.359876 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.359449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53800)
0810 11:44:50.359594 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.359668 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 11:44:50.360118 28348 leader_election.cc:334] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189): Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.363305 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.356953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:38591 (local address 127.0.0.1:39434)
0810 11:44:50.357619 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.357641 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.357811 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.358248 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.358254 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.358280 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.358587 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.358597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.359803 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.359811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.361321 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.361335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.362507 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.362514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.362695 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.362720 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.362831 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.362966 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":319,"threads_started":1}
I20260810 11:44:50.363375 28951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.357170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.20.93.189:38591)
0810 11:44:50.357893 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.357899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.357923 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:44:50.358021 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.358026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.358114 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.358223 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.358793 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.359646 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.361489 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.362341 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.362493 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.362600 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.362872 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.363011 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":388,"threads_started":1}
I20260810 11:44:50.364105 28168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "888c0f9f38b74bb0a073df3938343815" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0a8032ecd65c4c55b0ee3609108078e4" is_pre_election: true
I20260810 11:44:50.364851 28344 leader_election.cc:302] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 888c0f9f38b74bb0a073df3938343815; no voters: 0a8032ecd65c4c55b0ee3609108078e4, 144d9109aa714068a117b47cf32e9fca
I20260810 11:44:50.365180 28947 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:44:50.371977 28953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 144d9109aa714068a117b47cf32e9fca)
I20260810 11:44:50.372167 28953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } }
I20260810 11:44:50.372975 28953 leader_election.cc:288] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189), 888c0f9f38b74bb0a073df3938343815 (127.20.93.188:41177)
I20260810 11:44:50.374465 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.373119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53802)
0810 11:44:50.373848 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.374222 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":660,"threads_started":1}
W20260810 11:44:50.374811 28141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.376595 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.376234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53806)
0810 11:44:50.376378 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.376466 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 11:44:50.376816 28141 leader_election.cc:334] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189): Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.380448 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.373239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41177 (local address 127.0.0.1:41544)
0810 11:44:50.374274 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.374295 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.374469 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.374944 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.374948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.374975 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.375333 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.375342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.376829 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.376835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.378404 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.378417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.379633 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.379639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.379835 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.379860 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.379966 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.380099 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":399,"threads_started":1}
I20260810 11:44:50.380501 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.373418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41544 (local address 127.20.93.188:41177)
0810 11:44:50.374034 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.374040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.374423 (+ 383us) server_negotiation.cc:407] Connection header received
0810 11:44:50.374626 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.374631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.374704 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.374825 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.375510 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.376712 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.378556 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.379445 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.379598 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.379713 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.379992 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.380139 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":367,"threads_started":1}
I20260810 11:44:50.381141 28371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a8032ecd65c4c55b0ee3609108078e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "888c0f9f38b74bb0a073df3938343815" is_pre_election: true
I20260810 11:44:50.381431 28371 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a8032ecd65c4c55b0ee3609108078e4 in term 1.
I20260810 11:44:50.381879 28140 leader_election.cc:302] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a8032ecd65c4c55b0ee3609108078e4, 888c0f9f38b74bb0a073df3938343815; no voters: 144d9109aa714068a117b47cf32e9fca
I20260810 11:44:50.382148 28953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 11:44:50.382289 28953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 144d9109aa714068a117b47cf32e9fca)
I20260810 11:44:50.382364 28953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:44:50.385083 28953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } }
I20260810 11:44:50.385825 28953 leader_election.cc:288] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 election: Requested vote from peers 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189), 888c0f9f38b74bb0a073df3938343815 (127.20.93.188:41177)
I20260810 11:44:50.386444 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.386008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53818)
0810 11:44:50.386164 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.386269 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 11:44:50.386754 28371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a8032ecd65c4c55b0ee3609108078e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "888c0f9f38b74bb0a073df3938343815"
I20260810 11:44:50.386961 28371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:44:50.388837 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.388437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53832)
0810 11:44:50.388597 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.388699 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:44:50.389076 28141 leader_election.cc:334] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189): Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
I20260810 11:44:50.389564 28371 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a8032ecd65c4c55b0ee3609108078e4 in term 2.
I20260810 11:44:50.389962 28140 leader_election.cc:302] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a8032ecd65c4c55b0ee3609108078e4, 888c0f9f38b74bb0a073df3938343815; no voters: 144d9109aa714068a117b47cf32e9fca
I20260810 11:44:50.390259 28953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:44:50.390653 28953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [term 2 LEADER]: Becoming Leader. State: Replica: 0a8032ecd65c4c55b0ee3609108078e4, State: Running, Role: LEADER
I20260810 11:44:50.391019 28953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } }
I20260810 11:44:50.392937 28959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a8032ecd65c4c55b0ee3609108078e4. Latest consensus state: current_term: 2 leader_uuid: "0a8032ecd65c4c55b0ee3609108078e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:50.393359 28959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:50.394069 28961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:50.397750 28961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:50.398661 28961 catalog_manager.cc:1259] Loaded cluster ID: 04def2b6653248a2b8f356722d9a7daa
I20260810 11:44:50.398756 28961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:50.399971 28961 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:50.400797 28961 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Loaded TSK: 0
I20260810 11:44:50.401711 28961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:44:50.442373 28156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:50.449788 28156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:50.454290 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.453774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53856)
0810 11:44:50.453971 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.454098 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 11:44:50.454435 28371 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [term 2 FOLLOWER]: Refusing update from remote peer 0a8032ecd65c4c55b0ee3609108078e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 11:44:50.455240 28959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 11:44:50.455492 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.455055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53868)
0810 11:44:50.455265 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.455340 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 11:44:50.455761 28141 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 -> Peer 144d9109aa714068a117b47cf32e9fca (127.20.93.190:34189): Couldn't send request to peer 144d9109aa714068a117b47cf32e9fca. Status: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 11:44:50.458293 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a8032ecd65c4c55b0ee3609108078e4. Latest consensus state: current_term: 2 leader_uuid: "0a8032ecd65c4c55b0ee3609108078e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:50.458674 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:50.460140 28959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0a8032ecd65c4c55b0ee3609108078e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:50.460472 28959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:50.460788 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0a8032ecd65c4c55b0ee3609108078e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "144d9109aa714068a117b47cf32e9fca" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34189 } } peers { permanent_uuid: "0a8032ecd65c4c55b0ee3609108078e4" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 38591 } attrs { promote: false } } peers { permanent_uuid: "888c0f9f38b74bb0a073df3938343815" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41177 } attrs { promote: false } } }
I20260810 11:44:50.461046 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888c0f9f38b74bb0a073df3938343815 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:44:50.461274 28963 mvcc.cc:205] Tried to move back new op lower bound from 7316939941693526016 to 7316939941445312512. Current Snapshot: MvccSnapshot[applied={T|T < 7316939941693526016}]
I20260810 11:44:50.479648 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.469171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44303 (local address 127.0.0.1:40140)
0810 11:44:50.469439 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.469463 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.469640 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.470831 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.470836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.470857 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.471073 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.471086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.473084 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.473092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.475798 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.475821 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.478541 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.478552 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.478852 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.478907 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.478994 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.479206 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260810 11:44:50.480855 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.469353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.20.93.129:44303)
0810 11:44:50.470355 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.470363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.470382 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:44:50.470455 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.470460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.470547 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.470656 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.471451 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.472912 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.476022 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.477870 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.479984 (+ 2114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.480140 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.480214 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.480441 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":755,"threads_started":1}
I20260810 11:44:50.481856 28483 tablet_service.cc:1467] Processing CreateTablet for tablet 3099ec27f99e4a73a837840d302da8a1 (DEFAULT_TABLE table=test-table [id=751a44bce51d476c8161c9c3f79c7327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:50.482421 28483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3099ec27f99e4a73a837840d302da8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:50.490001 28972 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Bootstrap starting.
I20260810 11:44:50.490108 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.469693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44411 (local address 127.0.0.1:43172)
0810 11:44:50.470041 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.470062 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.470199 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.483575 (+ 13376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.483579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.483603 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.483912 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.483923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.485261 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.485266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.487395 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.487410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.489154 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.489160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.489418 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.489481 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.489625 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.489782 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":46}
I20260810 11:44:50.490376 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.470633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.20.93.131:39623)
0810 11:44:50.482110 (+ 11477us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.482119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.482145 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:44:50.482247 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.482257 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.482384 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.482514 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.483445 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.484533 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.487295 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.488651 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.488781 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.488894 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.489886 (+ 992us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.490054 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11303,"thread_start_us":11163,"threads_started":1}
I20260810 11:44:50.490779 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.470450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.20.93.130:44411)
0810 11:44:50.483254 (+ 12804us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.483261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.483281 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:44:50.483359 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.483368 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.483448 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.483559 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.484059 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.485101 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.487620 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.488933 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.489361 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.489484 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.490315 (+ 831us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.490487 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12661,"thread_start_us":13216,"threads_started":1}
I20260810 11:44:50.490845 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.470145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:39623 (local address 127.0.0.1:35588)
0810 11:44:50.470724 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.470744 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.470849 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.482789 (+ 11940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.482797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.482835 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.483254 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.483268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.484665 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.484670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.487082 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.487098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.489057 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.489063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.489266 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.489334 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.489425 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.489621 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":62,"thread_start_us":975,"threads_started":1}
I20260810 11:44:50.491611 28641 tablet_service.cc:1467] Processing CreateTablet for tablet 3099ec27f99e4a73a837840d302da8a1 (DEFAULT_TABLE table=test-table [id=751a44bce51d476c8161c9c3f79c7327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:50.491703 28796 tablet_service.cc:1467] Processing CreateTablet for tablet 3099ec27f99e4a73a837840d302da8a1 (DEFAULT_TABLE table=test-table [id=751a44bce51d476c8161c9c3f79c7327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:50.492105 28641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3099ec27f99e4a73a837840d302da8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:50.492118 28796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3099ec27f99e4a73a837840d302da8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:50.492359 28972 tablet_bootstrap.cc:654] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:50.498311 28972 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: No bootstrap required, opened a new log
I20260810 11:44:50.498523 28972 ts_tablet_manager.cc:1397] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260810 11:44:50.499357 28972 raft_consensus.cc:348] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.499619 28972 raft_consensus.cc:374] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:50.499722 28972 raft_consensus.cc:729] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc17f0a5c6043b8a6131b6641f19ef0, State: Initialized, Role: FOLLOWER
I20260810 11:44:50.499963 28972 consensus_queue.cc:260] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.502202 28972 ts_tablet_manager.cc:1428] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 11:44:50.502833 28974 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Bootstrap starting.
I20260810 11:44:50.503935 28976 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Bootstrap starting.
I20260810 11:44:50.504088 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.503047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:37497)
0810 11:44:50.503613 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.503776 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":369,"threads_started":1}
I20260810 11:44:50.504715 28974 tablet_bootstrap.cc:654] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:50.507136 28976 tablet_bootstrap.cc:654] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:50.508068 28556 heartbeater.cc:503] Master 127.20.93.189:38591 was elected leader, sending a full tablet report...
I20260810 11:44:50.508678 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.504913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:47179)
0810 11:44:50.508438 (+ 3525us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.508516 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3228}
I20260810 11:44:50.510818 28976 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: No bootstrap required, opened a new log
I20260810 11:44:50.510983 28976 ts_tablet_manager.cc:1397] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260810 11:44:50.511641 28976 raft_consensus.cc:348] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.511844 28976 raft_consensus.cc:374] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:50.511960 28976 raft_consensus.cc:729] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3408f93054024ae38fdb58991acd437b, State: Initialized, Role: FOLLOWER
I20260810 11:44:50.512207 28976 consensus_queue.cc:260] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.513046 28976 ts_tablet_manager.cc:1428] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 11:44:50.513960 28974 tablet_bootstrap.cc:492] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: No bootstrap required, opened a new log
I20260810 11:44:50.514128 28974 ts_tablet_manager.cc:1397] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260810 11:44:50.514273 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.513222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:43797)
0810 11:44:50.513813 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.513995 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":309,"threads_started":1}
I20260810 11:44:50.515024 28974 raft_consensus.cc:348] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.515291 28974 raft_consensus.cc:374] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:50.515420 28974 raft_consensus.cc:729] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b0558dedac4a24bba11bcf72a51030, State: Initialized, Role: FOLLOWER
I20260810 11:44:50.515679 28974 consensus_queue.cc:260] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.516484 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.516043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:60111)
0810 11:44:50.516258 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.516342 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 11:44:50.516997 28716 heartbeater.cc:503] Master 127.20.93.189:38591 was elected leader, sending a full tablet report...
I20260810 11:44:50.518442 28974 ts_tablet_manager.cc:1428] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260810 11:44:50.519712 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.518670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:42575)
0810 11:44:50.519304 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.519460 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":307,"threads_started":1}
I20260810 11:44:50.521368 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.520345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:40117)
0810 11:44:50.520925 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.521081 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":290,"threads_started":1}
I20260810 11:44:50.521376 28868 heartbeater.cc:503] Master 127.20.93.189:38591 was elected leader, sending a full tablet report...
I20260810 11:44:50.571413 28984 raft_consensus.cc:482] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:50.571693 28984 raft_consensus.cc:504] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.573459 28984 leader_election.cc:288] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303), 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
W20260810 11:44:50.577195 28869 tablet.cc:2369] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:50.580704 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.573931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:38251 (local address 127.20.93.130:44411)
0810 11:44:50.574283 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.574289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.574874 (+ 585us) server_negotiation.cc:407] Connection header received
0810 11:44:50.575028 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.575033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.575109 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.575206 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.575942 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.576912 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.578858 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.579711 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.579864 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.579952 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.580277 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.580418 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260810 11:44:50.580716 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.573862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:58285 (local address 127.20.93.129:44303)
0810 11:44:50.574228 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.574234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.574497 (+ 263us) server_negotiation.cc:407] Connection header received
0810 11:44:50.574712 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.574718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.574812 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.574907 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:50.575779 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.576623 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.578810 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.579753 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.579945 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.580026 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.580324 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.580460 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260810 11:44:50.580755 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.573858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44303 (local address 127.20.93.131:58285)
0810 11:44:50.574374 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.574393 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.574547 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.575063 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.575068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.575097 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.575579 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.575595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.576756 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.576762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.578624 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.578637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.579982 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.579990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.580132 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.580156 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.580277 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.580418 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":4}
I20260810 11:44:50.580760 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.573943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44411 (local address 127.20.93.131:38251)
0810 11:44:50.574760 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.574778 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:50.574909 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:50.575237 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:50.575242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:50.575269 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:50.575738 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.575753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.577071 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.577076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.578682 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:50.578695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.579902 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.579909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:50.580111 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.580135 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:44:50.580216 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 11:44:50.580406 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":1}
I20260810 11:44:50.581805 28663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3099ec27f99e4a73a837840d302da8a1" candidate_uuid: "13b0558dedac4a24bba11bcf72a51030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3408f93054024ae38fdb58991acd437b" is_pre_election: true
I20260810 11:44:50.581770 28506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3099ec27f99e4a73a837840d302da8a1" candidate_uuid: "13b0558dedac4a24bba11bcf72a51030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" is_pre_election: true
I20260810 11:44:50.582068 28663 raft_consensus.cc:2454] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b0558dedac4a24bba11bcf72a51030 in term 0.
I20260810 11:44:50.582161 28506 raft_consensus.cc:2454] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b0558dedac4a24bba11bcf72a51030 in term 0.
I20260810 11:44:50.582728 28745 leader_election.cc:302] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b0558dedac4a24bba11bcf72a51030, 3408f93054024ae38fdb58991acd437b; no voters:
I20260810 11:44:50.583158 28984 raft_consensus.cc:2781] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:50.583304 28984 raft_consensus.cc:482] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:50.583410 28984 raft_consensus.cc:3037] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:50.585903 28984 raft_consensus.cc:504] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.586519 28984 leader_election.cc:288] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [CANDIDATE]: Term 1 election: Requested vote from peers 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303), 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411)
W20260810 11:44:50.587006 28717 tablet.cc:2369] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:50.587123 28506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3099ec27f99e4a73a837840d302da8a1" candidate_uuid: "13b0558dedac4a24bba11bcf72a51030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0"
I20260810 11:44:50.587190 28663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3099ec27f99e4a73a837840d302da8a1" candidate_uuid: "13b0558dedac4a24bba11bcf72a51030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3408f93054024ae38fdb58991acd437b"
I20260810 11:44:50.587404 28506 raft_consensus.cc:3037] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:50.587412 28663 raft_consensus.cc:3037] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:50.589766 28663 raft_consensus.cc:2454] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b0558dedac4a24bba11bcf72a51030 in term 1.
I20260810 11:44:50.589934 28506 raft_consensus.cc:2454] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b0558dedac4a24bba11bcf72a51030 in term 1.
I20260810 11:44:50.590301 28745 leader_election.cc:302] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b0558dedac4a24bba11bcf72a51030, 3408f93054024ae38fdb58991acd437b; no voters:
I20260810 11:44:50.590656 28984 raft_consensus.cc:2781] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:50.591105 28984 raft_consensus.cc:686] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 1 LEADER]: Becoming Leader. State: Replica: 13b0558dedac4a24bba11bcf72a51030, State: Running, Role: LEADER
I20260810 11:44:50.591733 28984 consensus_queue.cc:237] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } }
I20260810 11:44:50.592221 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.591739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:51169)
0810 11:44:50.591911 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.592029 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 11:44:50.595506 28155 catalog_manager.cc:5495] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "13b0558dedac4a24bba11bcf72a51030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13b0558dedac4a24bba11bcf72a51030" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 39623 } health_report { overall_health: HEALTHY } } }
I20260810 11:44:50.648566 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.639210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.20.93.131:39623)
0810 11:44:50.639470 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.639478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.642021 (+ 2543us) server_negotiation.cc:407] Connection header received
0810 11:44:50.642253 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.642260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.642368 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.642535 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:50.643070 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.643874 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.646144 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.646675 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.646809 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.647353 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:50.647933 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:50.648061 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.648217 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.648299 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:44:50.653556 28663 raft_consensus.cc:1261] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Refusing update from remote peer 13b0558dedac4a24bba11bcf72a51030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:50.653944 28506 raft_consensus.cc:1261] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Refusing update from remote peer 13b0558dedac4a24bba11bcf72a51030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:50.654484 28988 consensus_queue.cc:1035] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3408f93054024ae38fdb58991acd437b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:50.654959 28984 consensus_queue.cc:1035] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc17f0a5c6043b8a6131b6641f19ef0" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:50.658185 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.657733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:56647)
0810 11:44:50.657880 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.657997 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 11:44:50.658808 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.657313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:54109)
0810 11:44:50.658090 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.658207 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":664}
I20260810 11:44:50.666633 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.665992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.130:47733)
0810 11:44:50.666294 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.666409 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260810 11:44:50.669697 28992 mvcc.cc:205] Tried to move back new op lower bound from 7316939942508683264 to 7316939942267686912. Current Snapshot: MvccSnapshot[applied={T|T < 7316939942508683264}]
I20260810 11:44:50.674060 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.673078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:50429)
0810 11:44:50.673741 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.673873 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539}
I20260810 11:44:50.676717 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.676193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.131:47365)
0810 11:44:50.676373 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.676497 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 11:44:50.678063 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.673005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.20.93.129:37015)
0810 11:44:50.674062 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.674201 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":908}
I20260810 11:44:50.688125 28156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:50.688465 28156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:50.697487 28156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a8032ecd65c4c55b0ee3609108078e4: Sending DeleteTablet for 3 replicas of tablet 3099ec27f99e4a73a837840d302da8a1
I20260810 11:44:50.698473 28483 tablet_service.cc:1514] Processing DeleteTablet for tablet 3099ec27f99e4a73a837840d302da8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:50 UTC) from {username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:40140
I20260810 11:44:50.699262 29001 tablet_replica.cc:330] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: stopping tablet replica
I20260810 11:44:50.699481 29001 raft_consensus.cc:2229] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:50.699822 29001 raft_consensus.cc:2258] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:50.699836 28641 tablet_service.cc:1514] Processing DeleteTablet for tablet 3099ec27f99e4a73a837840d302da8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:50 UTC) from {username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:43172
I20260810 11:44:50.700507 29003 tablet_replica.cc:330] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: stopping tablet replica
I20260810 11:44:50.700697 29003 raft_consensus.cc:2229] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:50.701045 29003 raft_consensus.cc:2258] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:50.701144 28796 tablet_service.cc:1514] Processing DeleteTablet for tablet 3099ec27f99e4a73a837840d302da8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:50 UTC) from {username='oryx', principal='oryx/127.20.93.189@KRBTEST.COM'} at 127.0.0.1:35588
I20260810 11:44:50.702502 29003 ts_tablet_manager.cc:1905] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:50.705701 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 28398
I20260810 11:44:50.706578 29001 ts_tablet_manager.cc:1905] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:50.708062 29004 tablet_replica.cc:330] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: stopping tablet replica
I20260810 11:44:50.708262 29004 raft_consensus.cc:2229] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:50.708952 29004 raft_consensus.cc:2258] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:50.710033 29003 ts_tablet_manager.cc:1918] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:50.710172 29003 log.cc:1196] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/3099ec27f99e4a73a837840d302da8a1
I20260810 11:44:50.710659 29003 ts_tablet_manager.cc:1939] T 3099ec27f99e4a73a837840d302da8a1 P 3408f93054024ae38fdb58991acd437b: Deleting consensus metadata
I20260810 11:44:50.711884 28140 catalog_manager.cc:4841] TS 3408f93054024ae38fdb58991acd437b (127.20.93.130:44411): tablet 3099ec27f99e4a73a837840d302da8a1 (table test-table [id=751a44bce51d476c8161c9c3f79c7327]) successfully deleted
I20260810 11:44:50.714020 29001 ts_tablet_manager.cc:1918] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:50.714162 29001 log.cc:1196] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/3099ec27f99e4a73a837840d302da8a1
I20260810 11:44:50.714716 29001 ts_tablet_manager.cc:1939] T 3099ec27f99e4a73a837840d302da8a1 P 4dc17f0a5c6043b8a6131b6641f19ef0: Deleting consensus metadata
I20260810 11:44:50.715984 28139 catalog_manager.cc:4841] TS 4dc17f0a5c6043b8a6131b6641f19ef0 (127.20.93.129:44303): tablet 3099ec27f99e4a73a837840d302da8a1 (table test-table [id=751a44bce51d476c8161c9c3f79c7327]) successfully deleted
I20260810 11:44:50.723131 29004 ts_tablet_manager.cc:1905] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:50.730677 29004 ts_tablet_manager.cc:1918] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:50.730929 29004 log.cc:1196] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/3099ec27f99e4a73a837840d302da8a1
I20260810 11:44:50.731525 29004 ts_tablet_manager.cc:1939] T 3099ec27f99e4a73a837840d302da8a1 P 13b0558dedac4a24bba11bcf72a51030: Deleting consensus metadata
I20260810 11:44:50.732854 28139 catalog_manager.cc:4841] TS 13b0558dedac4a24bba11bcf72a51030 (127.20.93.131:39623): tablet 3099ec27f99e4a73a837840d302da8a1 (table test-table [id=751a44bce51d476c8161c9c3f79c7327]) successfully deleted
I20260810 11:44:50.736673 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.707389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.20.93.129:44303)
0810 11:44:50.707612 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.707621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.718098 (+ 10477us) server_negotiation.cc:407] Connection header received
0810 11:44:50.718301 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.718307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.718401 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.718516 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:50.720378 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.721661 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.723814 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.724562 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.724671 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.728713 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:50.728734 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:50.728757 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:50.728797 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:50.733215 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:50.734085 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:50.734098 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:50.734103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:50.734296 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:50.734776 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:50.734782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:50.734787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:50.735297 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:50.735637 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.735962 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.736213 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:44:50.737413 28549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:50.894016 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 28398
I20260810 11:44:50.919960 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 28566
I20260810 11:44:50.933703 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.933216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53870)
0810 11:44:50.933399 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.933508 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 11:44:50.935079 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.934583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34189 (local address 127.0.0.1:53882)
0810 11:44:50.934808 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:50.934908 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 11:44:50.936898 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:50.920866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.20.93.130:44411)
0810 11:44:50.921077 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:50.921084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:50.921106 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:44:50.921180 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:50.921186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:50.921278 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:50.921390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:50.922801 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.923977 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.925178 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:50.925819 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:50.925957 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:50.928781 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:50.928805 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:50.928827 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:50.928878 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:50.933365 (+ 4487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:50.934379 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:50.934388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:50.934390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:50.934543 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:50.935027 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:50.935033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:50.935039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:50.935603 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:50.935966 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:50.936198 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 11:44:50.936476 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:44:50.937876 28709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:51.032837 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 28566
I20260810 11:44:51.055023 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 28727
I20260810 11:44:51.073313 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:51.055828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.20.93.131:39623)
0810 11:44:51.055982 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:51.055986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:51.056078 (+ 92us) server_negotiation.cc:407] Connection header received
0810 11:44:51.056261 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:51.056265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:51.056371 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:51.056454 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:51.058575 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:51.059312 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:51.061182 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:51.061650 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:51.061749 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:51.065222 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:51.065237 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:51.065258 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:51.065297 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:51.068660 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:51.069988 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:51.070003 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:51.070009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:51.070307 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:51.070819 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:51.070827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:51.070832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:51.071429 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:51.071792 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:51.072330 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 11:44:51.072608 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 11:44:51.074520 28861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:51.209167 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 28727
I20260810 11:44:51.233439 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 28007
I20260810 11:44:51.234599 28190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:51.358358 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 28007
I20260810 11:44:51.383014 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 28117
I20260810 11:44:51.384028 28394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:51.501204 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 28117
2026-08-10T11:44:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10152 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:51 dist-test-slave-njxd krb5kdc[29014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:51 dist-test-slave-njxd krb5kdc[29014](info): set up 2 sockets
Aug 10 11:44:51 dist-test-slave-njxd krb5kdc[29014](info): commencing operation
krb5kdc: starting...
W20260810 11:44:52.664736 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:44:52 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:44:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab.
Entry for principal oryx/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab.
I20260810 11:44:52.927321 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38779
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab
--principal=oryx/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:53.272367 29027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:53.272746 29027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:53.272903 29027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:53.280534 29027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:44:53.280707 29027 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:44:53.280807 29027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:53.280911 29027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:53.280997 29027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:44:53.281088 29027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:44:53.291675 29027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38779
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.190.keytab
--principal=oryx/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:53.293090 29027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:53.294540 29027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:53.307296 29035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:53.307377 29034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:53.307341 29038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:53.308097 29027 server_base.cc:1034] running on GCE node
I20260810 11:44:53.308890 29027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:53.310492 29027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:53.311755 29027 hybrid_clock.cc:648] HybridClock initialized: now 1786362293311674 us; error 47 us; skew 500 ppm
I20260810 11:44:53.312112 29027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:53 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362293, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:53.315845 29027 init.cc:377] Logged in from keytab as oryx/127.20.93.190@KRBTEST.COM (short username oryx)
I20260810 11:44:53.317548 29027 webserver.cc:460] Webserver started at http://127.20.93.190:43787/ using document root <none> and password file <none>
I20260810 11:44:53.318389 29027 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:53.318540 29027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:53.318899 29027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:53.322285 29027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "393d2ded76084279bee7f41a4e45b849"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "8b14408495023b27950985cb6e136f16"
server_key_iv: "6e26ac09683afa4bed1883306973f6b4"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.323143 29027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "393d2ded76084279bee7f41a4e45b849"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "8b14408495023b27950985cb6e136f16"
server_key_iv: "6e26ac09683afa4bed1883306973f6b4"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.328830 29027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:44:53.332515 29045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:53.334026 29027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:53.334216 29027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "393d2ded76084279bee7f41a4e45b849"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "8b14408495023b27950985cb6e136f16"
server_key_iv: "6e26ac09683afa4bed1883306973f6b4"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.334458 29027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:53.355856 29027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:53.360664 29027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:53.361012 29027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:53.383651 29027 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38779
I20260810 11:44:53.383674 29110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38779 every 8 connection(s)
I20260810 11:44:53.385162 29027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:44:53.389433 29111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:53.393543 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29027
I20260810 11:44:53.393807 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:44:53.394729 20854 external_mini_cluster.cc:1428] Setting key a13e6aaebf28110dbf23afe14439453c
I20260810 11:44:53.403702 29111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Bootstrap starting.
I20260810 11:44:53.407140 29111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:53.408317 29111 log.cc:824] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:44:53 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:53.411595 29111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: No bootstrap required, opened a new log
I20260810 11:44:53.419059 29114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:53.397197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.20.93.190:38779)
0810 11:44:53.398154 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:53.398167 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:53.398226 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:44:53.399965 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:53.399991 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:53.400524 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:53.401033 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:53.402690 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:53.404095 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:53.405372 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:53.406134 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:53.406352 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:53.410813 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:53.410823 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:53.410839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:53.410879 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:53.414385 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:53.415311 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:53.415319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:53.415330 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:53.415535 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:53.415939 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:53.415952 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:53.415954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:53.416563 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:53.416905 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:53.417381 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 11:44:53.417891 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":433,"threads_started":1}
I20260810 11:44:53.420411 29111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } }
I20260810 11:44:53.420909 29111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:53.421028 29111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393d2ded76084279bee7f41a4e45b849, State: Initialized, Role: FOLLOWER
I20260810 11:44:53.421754 29111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } }
I20260810 11:44:53.422015 29111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:44:53.422170 29111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:44:53.422384 29111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:53.424628 29111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } }
I20260810 11:44:53.425093 29111 leader_election.cc:302] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 393d2ded76084279bee7f41a4e45b849; no voters:
I20260810 11:44:53.425787 29111 leader_election.cc:288] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:44:53.425963 29116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:53.426671 29116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [term 1 LEADER]: Becoming Leader. State: Replica: 393d2ded76084279bee7f41a4e45b849, State: Running, Role: LEADER
I20260810 11:44:53.427310 29116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } }
I20260810 11:44:53.428236 29111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:44:53.430474 29117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "393d2ded76084279bee7f41a4e45b849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } } }
I20260810 11:44:53.430910 29117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:53.432008 29118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 393d2ded76084279bee7f41a4e45b849. Latest consensus state: current_term: 1 leader_uuid: "393d2ded76084279bee7f41a4e45b849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393d2ded76084279bee7f41a4e45b849" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38779 } } }
I20260810 11:44:53.432428 29118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849 [sys.catalog]: This master's current role is: LEADER
I20260810 11:44:53.433828 29121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:44:53.438582 29121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:44:53.449155 29121 catalog_manager.cc:1347] Generated new cluster ID: fb203007a1e844d184b53423a3c9c280
I20260810 11:44:53.449316 29121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:44:53.460799 29121 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:44:53.462350 29121 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:44:53.477541 29121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Generated new TSK 0
I20260810 11:44:53.478582 29121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
I20260810 11:44:53.551075 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38779
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:53.912173 29138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:53.912492 29138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:53.912637 29138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:53.920611 29138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:53.920754 29138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:53.920939 29138 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:53.921065 29138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:44:53.931640 29138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:53.933082 29138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:53.934600 29138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:53.949271 29144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:53.949303 29143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:53.949532 29146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:53.950153 29138 server_base.cc:1034] running on GCE node
I20260810 11:44:53.950888 29138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:53.951944 29138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:53.953236 29138 hybrid_clock.cc:648] HybridClock initialized: now 1786362293953158 us; error 52 us; skew 500 ppm
I20260810 11:44:53.953657 29138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:53 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362293, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:53.957226 29138 init.cc:377] Logged in from keytab as oryx/127.20.93.129@KRBTEST.COM (short username oryx)
I20260810 11:44:53.958915 29138 webserver.cc:460] Webserver started at http://127.20.93.129:35435/ using document root <none> and password file <none>
I20260810 11:44:53.959661 29138 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:53.959754 29138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:53.960048 29138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:53.963322 29138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "c0a74dbb3baa412782f6a07bdbf77978"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "5166c4c1b0bd24aa609356fc5d2612d5"
server_key_iv: "575331d0592cf6b11af6b4bda771df6f"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.964140 29138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "c0a74dbb3baa412782f6a07bdbf77978"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "5166c4c1b0bd24aa609356fc5d2612d5"
server_key_iv: "575331d0592cf6b11af6b4bda771df6f"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.969707 29138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 11:44:53.973364 29154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:53.974922 29138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:53.975057 29138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "c0a74dbb3baa412782f6a07bdbf77978"
format_stamp: "Formatted at 2026-08-10 11:44:53 on dist-test-slave-njxd"
server_key: "5166c4c1b0bd24aa609356fc5d2612d5"
server_key_iv: "575331d0592cf6b11af6b4bda771df6f"
server_key_version: "encryptionkey@0"
I20260810 11:44:53.975255 29138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:53.993717 29138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:53.998037 29138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:53.998288 29138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:54.001225 29138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:54.001322 29138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:54.001546 29138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:54.001715 29138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:54 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362293, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:54.025192 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.001676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:37651 (local address 127.20.93.190:38779)
0810 11:44:54.002210 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:54.002217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:54.003452 (+ 1235us) server_negotiation.cc:407] Connection header received
0810 11:44:54.005553 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:54.005559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:54.005702 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:54.005836 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:54.008114 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.009172 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.011983 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.012410 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.012553 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.017140 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:54.017145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:54.017148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:54.017184 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:54.020498 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.021762 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.021766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.021768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.021902 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.022518 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.022522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.022524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.022922 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:54.023072 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:54.024263 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0810 11:44:54.024610 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":255,"threads_started":1}
I20260810 11:44:54.025894 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.002341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.129:37651)
0810 11:44:54.003262 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:54.003310 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:54.005241 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:54.006084 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:54.006093 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:54.006904 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:54.007825 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.007856 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.009367 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.009392 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.011761 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.011779 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.012684 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.012692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.012965 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.013869 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:54.013896 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:54.016886 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:54.020722 (+ 3836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.020734 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.020758 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.021477 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.022023 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.022030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.022035 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.022388 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.023147 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:54.023154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:54.023716 (+ 562us) client_negotiation.cc:769] Sending connection context
0810 11:44:54.024081 (+ 365us) client_negotiation.cc:240] Negotiation successful
0810 11:44:54.024566 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":395,"threads_started":1}
I20260810 11:44:54.050941 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.033141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.129:35131)
0810 11:44:54.034068 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:54.034100 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:54.034296 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:54.034693 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:54.034701 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:54.035611 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:54.036158 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.036176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.037205 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.037215 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.039801 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.039831 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.040766 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.040776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.041055 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.041858 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:54.041879 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:54.042974 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:54.046985 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.046990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.046998 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.047673 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.048156 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.048163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.048168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.048353 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.049278 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:54.049286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:54.049560 (+ 274us) client_negotiation.cc:769] Sending connection context
0810 11:44:54.049782 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 11:44:54.050231 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":506,"threads_started":1}
I20260810 11:44:54.051748 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.032912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:35131 (local address 127.20.93.190:38779)
0810 11:44:54.033112 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:54.033120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:54.034220 (+ 1100us) server_negotiation.cc:407] Connection header received
0810 11:44:54.034455 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:54.034459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:54.034536 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:54.034623 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:54.036333 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.037027 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.040008 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.040545 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.040653 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.043178 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:54.043183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:54.043186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:54.043219 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:54.046830 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.047842 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.047847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.047850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.048021 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.048522 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.048530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.048534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.048980 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:54.049195 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:54.051046 (+ 1851us) server_negotiation.cc:299] Negotiation successful
0810 11:44:54.051339 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:44:54.058806 29138 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:35201
I20260810 11:44:54.058847 29284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:35201 every 8 connection(s)
I20260810 11:44:54.060058 29138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:44:54.060822 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29138
I20260810 11:44:54.061067 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:44:54.061925 20854 external_mini_cluster.cc:1428] Setting key 7b4ceeeb9a970e804ab97cd6770c38ff
I20260810 11:44:54.062469 29285 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38779
I20260810 11:44:54.063813 29285 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:54.064698 29285 heartbeater.cc:511] Master 127.20.93.190:38779 requested a full tablet report, sending...
I20260810 11:44:54.067214 29063 ts_manager.cc:194] Registered new tserver with Master: c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201)
I20260810 11:44:54.069416 29063 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:37651
WARNING: no policy specified for oryx/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
I20260810 11:44:54.125936 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38779
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:54.469451 29289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:54.469817 29289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:54.470000 29289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:54.477192 29289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:54.477337 29289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:54.477488 29289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:54.477671 29289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:44:54.487766 29289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:54.489113 29289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:54.490543 29289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:54.504921 29295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:54.504951 29294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:54.505384 29297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:54.505728 29289 server_base.cc:1034] running on GCE node
I20260810 11:44:54.506371 29289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:54.507320 29289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:54.508591 29289 hybrid_clock.cc:648] HybridClock initialized: now 1786362294508521 us; error 51 us; skew 500 ppm
I20260810 11:44:54.508913 29289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:54 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362294, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:54.512467 29289 init.cc:377] Logged in from keytab as oryx/127.20.93.130@KRBTEST.COM (short username oryx)
I20260810 11:44:54.514026 29289 webserver.cc:460] Webserver started at http://127.20.93.130:40943/ using document root <none> and password file <none>
I20260810 11:44:54.514803 29289 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:54.514943 29289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:54.515287 29289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:54.518501 29289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "3a64ff25bf5e42e7837102f0c95a55f8"
format_stamp: "Formatted at 2026-08-10 11:44:54 on dist-test-slave-njxd"
server_key: "1da31414e6e2b079f5915beaa82427b7"
server_key_iv: "c3e2be67bed73a8c0bc0a92983264f20"
server_key_version: "encryptionkey@0"
I20260810 11:44:54.519415 29289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "3a64ff25bf5e42e7837102f0c95a55f8"
format_stamp: "Formatted at 2026-08-10 11:44:54 on dist-test-slave-njxd"
server_key: "1da31414e6e2b079f5915beaa82427b7"
server_key_iv: "c3e2be67bed73a8c0bc0a92983264f20"
server_key_version: "encryptionkey@0"
I20260810 11:44:54.524927 29289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:44:54.529023 29304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:54.530649 29289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:54.530857 29289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "3a64ff25bf5e42e7837102f0c95a55f8"
format_stamp: "Formatted at 2026-08-10 11:44:54 on dist-test-slave-njxd"
server_key: "1da31414e6e2b079f5915beaa82427b7"
server_key_iv: "c3e2be67bed73a8c0bc0a92983264f20"
server_key_version: "encryptionkey@0"
I20260810 11:44:54.531104 29289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:54.545118 29289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:54.549419 29289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:54.549790 29289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:54.552686 29289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:54.552846 29289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:54.553031 29289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:54.553157 29289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:54 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362294, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:54.578506 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.553267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:56355 (local address 127.20.93.190:38779)
0810 11:44:54.553848 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:54.553854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:54.555032 (+ 1178us) server_negotiation.cc:407] Connection header received
0810 11:44:54.557623 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:54.557628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:54.557719 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:54.557844 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:54.560089 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.561622 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.564492 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.565310 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.565497 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.571223 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:54.571229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:54.571233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:54.571269 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:54.574649 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.575649 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.575657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.575660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.575800 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.576393 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.576399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.576402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.576789 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:54.576943 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:54.577859 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 11:44:54.578093 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":247,"threads_started":1}
I20260810 11:44:54.580123 29317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.553934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.130:56355)
0810 11:44:54.554850 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:54.554897 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:54.557327 (+ 2430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:54.558073 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:54.558081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:54.558876 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:54.559839 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.559875 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.561894 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.561905 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.564296 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.564317 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.565628 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.565642 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.565909 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.566736 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:54.566762 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:54.570891 (+ 4129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:54.574785 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.574795 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.574821 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.575516 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.575924 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.575929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.575937 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.576272 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.577026 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:54.577032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:54.577408 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 11:44:54.577774 (+ 366us) client_negotiation.cc:240] Negotiation successful
0810 11:44:54.578378 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":366,"threads_started":1}
I20260810 11:44:54.601676 29289 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:39411
I20260810 11:44:54.601722 29436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:39411 every 8 connection(s)
I20260810 11:44:54.602509 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.587514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.130:53531)
0810 11:44:54.588598 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:54.588613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:54.588722 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:54.589111 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:54.589114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:54.589538 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:54.589969 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.589980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.591767 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.591774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.593548 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:54.593560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.594359 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.594366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:54.594541 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.595275 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:54.595292 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:54.596005 (+ 713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:54.599365 (+ 3360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.599371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.599375 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.600006 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.600479 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:54.600484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:54.600488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:54.600705 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:54.601459 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:54.601464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:54.601726 (+ 262us) client_negotiation.cc:769] Sending connection context
0810 11:44:54.601912 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 11:44:54.602181 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":763,"threads_started":1}
I20260810 11:44:54.603170 29289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:44:54.603387 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:54.587570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:53531 (local address 127.20.93.190:38779)
0810 11:44:54.587818 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:54.587828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:54.588704 (+ 876us) server_negotiation.cc:407] Connection header received
0810 11:44:54.588828 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:54.588837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:54.588954 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:54.589111 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:54.590205 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.591596 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.593740 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:54.594166 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:54.594313 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:54.596198 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:54.596203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:54.596206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:54.596238 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:54.599171 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.600147 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.600150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.600153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.600339 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:54.600853 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:54.600857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:54.600862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:54.601338 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:54.601507 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:54.602930 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0810 11:44:54.603112 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:44:54.603981 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29289
I20260810 11:44:54.604164 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:44:54.604804 20854 external_mini_cluster.cc:1428] Setting key 37893e3eccc89a53dfbb71c0820e0d9d
I20260810 11:44:54.608829 29437 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38779
I20260810 11:44:54.609083 29437 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:54.610024 29437 heartbeater.cc:511] Master 127.20.93.190:38779 requested a full tablet report, sending...
I20260810 11:44:54.611814 29063 ts_manager.cc:194] Registered new tserver with Master: 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411)
I20260810 11:44:54.612697 29063 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:56355
WARNING: no policy specified for oryx/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
I20260810 11:44:54.664346 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38779
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:44:55.006322 29441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:44:55.006601 29441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:44:55.006701 29441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:44:55.013765 29441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:44:55.013875 29441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:44:55.013957 29441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:44:55.014037 29441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:44:55.023658 29441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.93.190:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:44:55.024922 29441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:44:55.026347 29441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:44:55.040309 29446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:55.040352 29450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:44:55.040308 29447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:44:55.041718 29441 server_base.cc:1034] running on GCE node
I20260810 11:44:55.042441 29441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:44:55.043475 29441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:44:55.044736 29441 hybrid_clock.cc:648] HybridClock initialized: now 1786362295044662 us; error 47 us; skew 500 ppm
I20260810 11:44:55.045084 29441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:44:55 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362295, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:44:55.048502 29441 init.cc:377] Logged in from keytab as oryx/127.20.93.131@KRBTEST.COM (short username oryx)
I20260810 11:44:55.050096 29441 webserver.cc:460] Webserver started at http://127.20.93.131:33401/ using document root <none> and password file <none>
I20260810 11:44:55.051072 29441 fs_manager.cc:359] Metadata directory not provided
I20260810 11:44:55.051220 29441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:44:55.051554 29441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:44:55.055120 29441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "6de50fc4657448318ef93d556713c647"
format_stamp: "Formatted at 2026-08-10 11:44:55 on dist-test-slave-njxd"
server_key: "dd15fedd246073d8120aa16ead7394c4"
server_key_iv: "b59df36b17118f0e3da67cc3631207c3"
server_key_version: "encryptionkey@0"
I20260810 11:44:55.055940 29441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "6de50fc4657448318ef93d556713c647"
format_stamp: "Formatted at 2026-08-10 11:44:55 on dist-test-slave-njxd"
server_key: "dd15fedd246073d8120aa16ead7394c4"
server_key_iv: "b59df36b17118f0e3da67cc3631207c3"
server_key_version: "encryptionkey@0"
I20260810 11:44:55.061455 29441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:44:55.065120 29457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:55.066529 29441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:55.066735 29441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "6de50fc4657448318ef93d556713c647"
format_stamp: "Formatted at 2026-08-10 11:44:55 on dist-test-slave-njxd"
server_key: "dd15fedd246073d8120aa16ead7394c4"
server_key_iv: "b59df36b17118f0e3da67cc3631207c3"
server_key_version: "encryptionkey@0"
I20260810 11:44:55.066970 29441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:44:55.073311 29285 heartbeater.cc:503] Master 127.20.93.190:38779 was elected leader, sending a full tablet report...
I20260810 11:44:55.087818 29441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:44:55.091586 29441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:44:55.091816 29441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:44:55.094456 29441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:44:55.094584 29441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:44:55.094786 29441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:44:55.094863 29441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:44:55 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362295, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:55.119719 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:55.094862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:50789 (local address 127.20.93.190:38779)
0810 11:44:55.095061 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:55.095065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:55.096937 (+ 1872us) server_negotiation.cc:407] Connection header received
0810 11:44:55.099311 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:55.099317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:55.099421 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:55.099485 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:55.102174 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.103573 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.106110 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.106908 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.107062 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:55.111384 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:55.111393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:55.111399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:55.111447 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:55.115728 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:55.116436 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:55.116444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:55.116448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:55.116676 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:55.117104 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:55.117112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:55.117118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:55.117609 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:55.117790 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:55.118875 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0810 11:44:55.119176 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:44:55.120240 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:55.095488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.131:50789)
0810 11:44:55.096760 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:55.096822 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:55.098916 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:55.100068 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:55.100082 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:55.100965 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:55.101940 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:55.101979 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.103759 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.103766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:55.105910 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:55.105926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.107158 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.107164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:55.107349 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:55.108095 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:55.108117 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:55.111068 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:55.115887 (+ 4819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:55.115894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:55.115906 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:55.116306 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:55.116797 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:55.116800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:55.116802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:55.116987 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:55.117768 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:55.117779 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:55.118259 (+ 480us) client_negotiation.cc:769] Sending connection context
0810 11:44:55.118693 (+ 434us) client_negotiation.cc:240] Negotiation successful
0810 11:44:55.119081 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":411,"threads_started":1}
I20260810 11:44:55.143997 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:55.126413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:40325 (local address 127.20.93.190:38779)
0810 11:44:55.126637 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:55.126646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:55.127221 (+ 575us) server_negotiation.cc:407] Connection header received
0810 11:44:55.127366 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:55.127374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:55.127475 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:55.127606 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:55.129231 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.130553 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.133327 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.133813 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.133958 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:55.136681 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:55.136691 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:55.136698 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:55.136749 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:55.141113 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:55.141971 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:55.141978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:55.141982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:55.142131 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:55.142510 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:55.142515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:55.142519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:55.142817 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:55.142971 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:55.143383 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 11:44:55.143617 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:44:55.144109 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:55.126424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.20.93.131:40325)
0810 11:44:55.127124 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:55.127140 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:55.127250 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:55.127616 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:55.127619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:55.128303 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:55.128956 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:55.128978 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.130772 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.130778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:55.133146 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:55.133168 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.134027 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.134036 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:55.134211 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:55.135186 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:55.135215 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:55.136450 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:55.141272 (+ 4822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:55.141276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:55.141279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:55.141819 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:55.142259 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:55.142264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:55.142268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:55.142400 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:55.142946 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:55.142951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:55.143138 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 11:44:55.143282 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 11:44:55.143632 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":391,"threads_started":1}
I20260810 11:44:55.157373 29441 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:32989
I20260810 11:44:55.157403 29589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:32989 every 8 connection(s)
I20260810 11:44:55.158607 29441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:44:55.160706 29590 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38779
I20260810 11:44:55.160897 29590 heartbeater.cc:464] Registering TS with master...
I20260810 11:44:55.161566 29590 heartbeater.cc:511] Master 127.20.93.190:38779 requested a full tablet report, sending...
I20260810 11:44:55.162297 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29441
I20260810 11:44:55.162483 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:44:55.163077 20854 external_mini_cluster.cc:1428] Setting key f73fd4f70e4a59f238208b448759beee
I20260810 11:44:55.163055 29063 ts_manager.cc:194] Registered new tserver with Master: 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:55.164189 29063 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:50789
I20260810 11:44:55.166083 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:44:55 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362295, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:44:55 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:44:55 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:44:55.615312 29437 heartbeater.cc:503] Master 127.20.93.190:38779 was elected leader, sending a full tablet report...
I20260810 11:44:56.166723 29590 heartbeater.cc:503] Master 127.20.93.190:38779 was elected leader, sending a full tablet report...
W20260810 11:44:56.345955 29598 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.128s user 0.000s sys 0.019s
W20260810 11:44:56.347101 29315 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:55.212813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.20.93.190:38779)
0810 11:44:55.213004 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:55.213024 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:55.213367 (+ 343us) server_negotiation.cc:407] Connection header received
0810 11:44:55.213497 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:55.213501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:55.213570 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:55.213723 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:55.215028 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.215761 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.216983 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:55.217423 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:55.217517 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.346874 (+1129357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42884: BlockingRecv error: recv got EOF from 127.0.0.1:42884 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 11:44:56 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362295, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:56.364948 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.350886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.20.93.190:38779)
0810 11:44:56.351132 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.351139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.351423 (+ 284us) server_negotiation.cc:407] Connection header received
0810 11:44:56.351635 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.351641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.351720 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.351835 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:56.353140 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.353950 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.355244 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.355673 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.355820 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.359751 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:56.359756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:56.359759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:56.359793 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:56.362337 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:56.362985 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:56.362993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:56.362996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:56.363132 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:56.363508 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:56.363514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:56.363517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:56.363812 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:56.364084 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.364379 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.364612 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:44:56.369103 29063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:56.376346 29063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:56.413552 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.390967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:32989 (local address 127.0.0.1:38722)
0810 11:44:56.403563 (+ 12596us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.403584 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.403733 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.405117 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.405125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.405159 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:56.405744 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.405759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.408010 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.408028 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.410660 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.410677 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.412449 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.412456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.412670 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.412737 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.412867 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.413104 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12431,"mutex_wait_us":1,"thread_start_us":12855,"threads_started":1}
I20260810 11:44:56.414672 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.391332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.20.93.131:32989)
0810 11:44:56.404408 (+ 13076us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.404418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.404447 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:44:56.404549 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.404559 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.404818 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.405984 (+ 1166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:56.406250 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.407828 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.410887 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.411894 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.411991 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.412162 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.414163 (+ 2001us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.414355 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12880,"thread_start_us":13269,"threads_started":1}
I20260810 11:44:56.416312 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.390464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:39411 (local address 127.0.0.1:38838)
0810 11:44:56.403354 (+ 12890us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.403400 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.403614 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.406096 (+ 2482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.406103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.406130 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:56.407508 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.407523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.409706 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.409712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.413489 (+ 3777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.413503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.415509 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.415514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.415674 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.415709 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.415809 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.415957 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12597,"thread_start_us":12419,"threads_started":1}
I20260810 11:44:56.416456 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.389647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.20.93.130:39411)
0810 11:44:56.390280 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.390289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.403551 (+ 13262us) server_negotiation.cc:407] Connection header received
0810 11:44:56.403846 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.403855 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.404179 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.404401 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:56.407913 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.409495 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.413721 (+ 4226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.415099 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.415194 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.415358 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.415912 (+ 554us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.416123 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":340,"threads_started":1}
I20260810 11:44:56.418999 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.404103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.20.93.129:35201)
0810 11:44:56.405251 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.405263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.405930 (+ 667us) server_negotiation.cc:407] Connection header received
0810 11:44:56.406087 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.406094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.406289 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.406468 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:56.408255 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.409876 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.416339 (+ 6463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.417908 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.418071 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.418282 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.418541 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.418703 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":817,"threads_started":1}
I20260810 11:44:56.418985 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.404046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35201 (local address 127.0.0.1:43996)
0810 11:44:56.405795 (+ 1749us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.405820 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.405949 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.406664 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.406669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.406693 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:56.408032 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.408045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.410091 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.410096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.416127 (+ 6031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.416142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.418138 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.418144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.418325 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.418357 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.418468 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.418609 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1610,"mutex_wait_us":1,"thread_start_us":1411,"threads_started":1}
I20260810 11:44:56.418823 29511 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 (DEFAULT_TABLE table=test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.420457 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5cc4edd01f4053805cb432553c5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.420442 29362 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 (DEFAULT_TABLE table=test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.421926 29362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5cc4edd01f4053805cb432553c5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.424625 29210 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 (DEFAULT_TABLE table=test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.425810 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5cc4edd01f4053805cb432553c5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.435786 29619 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Bootstrap starting.
I20260810 11:44:56.438632 29621 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Bootstrap starting.
I20260810 11:44:56.441898 29619 tablet_bootstrap.cc:654] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.442337 29620 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Bootstrap starting.
I20260810 11:44:56.442983 29621 tablet_bootstrap.cc:654] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.443503 29619 log.cc:824] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:56.444115 29621 log.cc:824] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:56.445703 29620 tablet_bootstrap.cc:654] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.446938 29620 log.cc:824] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:44:56.447125 29619 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: No bootstrap required, opened a new log
I20260810 11:44:56.447278 29621 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: No bootstrap required, opened a new log
I20260810 11:44:56.447422 29619 ts_tablet_manager.cc:1397] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 11:44:56.447657 29621 ts_tablet_manager.cc:1397] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 11:44:56.450127 29620 tablet_bootstrap.cc:492] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: No bootstrap required, opened a new log
I20260810 11:44:56.450380 29620 ts_tablet_manager.cc:1397] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260810 11:44:56.455662 29621 raft_consensus.cc:348] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.456044 29621 raft_consensus.cc:374] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.456153 29621 raft_consensus.cc:729] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a74dbb3baa412782f6a07bdbf77978, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.456786 29621 consensus_queue.cc:260] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.458415 29621 ts_tablet_manager.cc:1428] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 11:44:56.460106 29619 raft_consensus.cc:348] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.460551 29619 raft_consensus.cc:374] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.460660 29619 raft_consensus.cc:729] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de50fc4657448318ef93d556713c647, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.460728 29620 raft_consensus.cc:348] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.461192 29620 raft_consensus.cc:374] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.461310 29620 raft_consensus.cc:729] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.461247 29619 consensus_queue.cc:260] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.461912 29620 consensus_queue.cc:260] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.463671 29620 ts_tablet_manager.cc:1428] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260810 11:44:56.465413 29619 ts_tablet_manager.cc:1428] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Time spent starting tablet: real 0.018s user 0.009s sys 0.004s
I20260810 11:44:56.528491 29630 raft_consensus.cc:482] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:56.528903 29630 raft_consensus.cc:504] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.531075 29630 leader_election.cc:288] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:56.538368 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.531717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:48471 (local address 127.20.93.131:32989)
0810 11:44:56.532073 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.532083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.532462 (+ 379us) server_negotiation.cc:407] Connection header received
0810 11:44:56.532702 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.532708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.532776 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.532874 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:56.533449 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.534255 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.536329 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.537181 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.537392 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.537477 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.537900 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.538036 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260810 11:44:56.538465 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.531391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:32989 (local address 127.20.93.130:48471)
0810 11:44:56.532336 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.532354 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.532554 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.532896 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.532901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.532936 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:56.533300 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.533309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.534392 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.534398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.536165 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.536181 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.537416 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.537423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.537641 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.537680 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.537870 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.538015 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":53,"thread_start_us":603,"threads_started":1}
I20260810 11:44:56.538743 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.531464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:40809 (local address 127.20.93.129:35201)
0810 11:44:56.531802 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.531812 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.532769 (+ 957us) server_negotiation.cc:407] Connection header received
0810 11:44:56.532992 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.532997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.533062 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.533170 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:56.533781 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.534568 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.536606 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.537546 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.537815 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.537899 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.538294 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.538441 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260810 11:44:56.539173 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.531483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35201 (local address 127.20.93.130:40809)
0810 11:44:56.532642 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.532662 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.532836 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.533191 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.533195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.533226 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:56.533611 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.533621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.534726 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.534732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.536422 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.536436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.537844 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.537852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.538090 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.538147 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.538266 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.538429 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":6,"thread_start_us":822,"threads_started":1}
I20260810 11:44:56.539662 29539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d5cc4edd01f4053805cb432553c5ba1" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647" is_pre_election: true
I20260810 11:44:56.540292 29539 raft_consensus.cc:2454] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:56.540400 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d5cc4edd01f4053805cb432553c5ba1" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978" is_pre_election: true
I20260810 11:44:56.540912 29238 raft_consensus.cc:2454] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:56.541011 29305 leader_election.cc:302] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, 6de50fc4657448318ef93d556713c647; no voters:
I20260810 11:44:56.541489 29630 raft_consensus.cc:2781] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:56.541697 29630 raft_consensus.cc:482] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:56.541821 29630 raft_consensus.cc:3037] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.544492 29630 raft_consensus.cc:504] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.545140 29630 leader_election.cc:288] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Requested vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:56.545465 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d5cc4edd01f4053805cb432553c5ba1" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978"
I20260810 11:44:56.545728 29238 raft_consensus.cc:3037] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.545692 29539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d5cc4edd01f4053805cb432553c5ba1" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647"
I20260810 11:44:56.545917 29539 raft_consensus.cc:3037] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.549151 29539 raft_consensus.cc:2454] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:56.549151 29238 raft_consensus.cc:2454] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:56.549638 29305 leader_election.cc:302] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, 6de50fc4657448318ef93d556713c647; no voters:
I20260810 11:44:56.549970 29630 raft_consensus.cc:2781] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:56.550452 29630 raft_consensus.cc:686] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Becoming Leader. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Running, Role: LEADER
I20260810 11:44:56.550943 29630 consensus_queue.cc:237] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.555800 29062 catalog_manager.cc:5495] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130). New cstate: current_term: 1 leader_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } health_report { overall_health: UNKNOWN } } }
W20260810 11:44:56.564145 29286 tablet.cc:2369] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:56.592636 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.587484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.20.93.130:39411)
0810 11:44:56.587718 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.587723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.587743 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:44:56.587929 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.587933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.588013 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.588143 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:56.588464 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.589170 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.590516 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.590979 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.591079 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.591466 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:56.592043 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:56.592150 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.592256 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.592326 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:44:56.600807 29539 raft_consensus.cc:1261] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Refusing update from remote peer 3a64ff25bf5e42e7837102f0c95a55f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:56.600849 29238 raft_consensus.cc:1261] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Refusing update from remote peer 3a64ff25bf5e42e7837102f0c95a55f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:56.601881 29633 consensus_queue.cc:1035] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:56.602267 29630 consensus_queue.cc:1035] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:44:56.612161 29438 tablet.cc:2369] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:44:56.618767 29637 mvcc.cc:205] Tried to move back new op lower bound from 7316939966867525632 to 7316939966676799488. Current Snapshot: MvccSnapshot[applied={T|T < 7316939966867525632}]
I20260810 11:44:56.619688 29636 mvcc.cc:205] Tried to move back new op lower bound from 7316939966867525632 to 7316939966676799488. Current Snapshot: MvccSnapshot[applied={T|T < 7316939966867525632}]
I20260810 11:44:56.622153 29638 mvcc.cc:205] Tried to move back new op lower bound from 7316939966867525632 to 7316939966676799488. Current Snapshot: MvccSnapshot[applied={T|T < 7316939966867525632}]
I20260810 11:44:56.634680 29063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:56.635025 29063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:56.639195 29063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Sending DeleteTablet for 3 replicas of tablet 6d5cc4edd01f4053805cb432553c5ba1
I20260810 11:44:56.640573 29362 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:56 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38838
I20260810 11:44:56.640616 29210 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:56 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43996
I20260810 11:44:56.640549 29511 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d5cc4edd01f4053805cb432553c5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:56 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260810 11:44:56.647804 29062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:56.649951 29062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:56.651310 29646 tablet_replica.cc:330] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: stopping tablet replica
I20260810 11:44:56.652494 29646 raft_consensus.cc:2229] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:56.653683 29646 raft_consensus.cc:2258] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:56.654735 29647 tablet_replica.cc:330] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: stopping tablet replica
I20260810 11:44:56.655623 29645 tablet_replica.cc:330] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: stopping tablet replica
I20260810 11:44:56.656116 29645 raft_consensus.cc:2229] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:56.656111 29647 raft_consensus.cc:2229] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:56.656661 29645 raft_consensus.cc:2258] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:56.657797 29646 ts_tablet_manager.cc:1905] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:56.658504 29647 raft_consensus.cc:2258] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:56.658751 29645 ts_tablet_manager.cc:1905] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:56.660511 29210 tablet_service.cc:1467] Processing CreateTablet for tablet 6652e83662b64c28bc20fe295b944c71 (DEFAULT_TABLE table=test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.660892 29362 tablet_service.cc:1467] Processing CreateTablet for tablet 6652e83662b64c28bc20fe295b944c71 (DEFAULT_TABLE table=test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.660987 29511 tablet_service.cc:1467] Processing CreateTablet for tablet 6652e83662b64c28bc20fe295b944c71 (DEFAULT_TABLE table=test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:56.661033 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6652e83662b64c28bc20fe295b944c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.661341 29362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6652e83662b64c28bc20fe295b944c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.661551 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6652e83662b64c28bc20fe295b944c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.666393 29645 ts_tablet_manager.cc:1918] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:56.666611 29645 log.cc:1196] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/6d5cc4edd01f4053805cb432553c5ba1
I20260810 11:44:56.667179 29645 ts_tablet_manager.cc:1939] T 6d5cc4edd01f4053805cb432553c5ba1 P 6de50fc4657448318ef93d556713c647: Deleting consensus metadata
I20260810 11:44:56.668372 29647 ts_tablet_manager.cc:1905] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:56.669181 29621 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Bootstrap starting.
I20260810 11:44:56.669158 29046 catalog_manager.cc:4841] TS 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989): tablet 6d5cc4edd01f4053805cb432553c5ba1 (table test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]) successfully deleted
I20260810 11:44:56.670898 29621 tablet_bootstrap.cc:654] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.671052 29619 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Bootstrap starting.
I20260810 11:44:56.672389 29620 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Bootstrap starting.
I20260810 11:44:56.672585 29619 tablet_bootstrap.cc:654] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.673825 29621 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: No bootstrap required, opened a new log
I20260810 11:44:56.674003 29621 ts_tablet_manager.cc:1397] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:44:56.674163 29620 tablet_bootstrap.cc:654] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.675261 29621 raft_consensus.cc:348] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.675582 29621 raft_consensus.cc:374] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.675709 29621 raft_consensus.cc:729] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a74dbb3baa412782f6a07bdbf77978, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.676044 29621 consensus_queue.cc:260] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.677321 29621 ts_tablet_manager.cc:1428] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 11:44:56.677479 29646 ts_tablet_manager.cc:1918] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:56.677721 29646 log.cc:1196] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/6d5cc4edd01f4053805cb432553c5ba1
I20260810 11:44:56.678382 29646 ts_tablet_manager.cc:1939] T 6d5cc4edd01f4053805cb432553c5ba1 P c0a74dbb3baa412782f6a07bdbf77978: Deleting consensus metadata
I20260810 11:44:56.679075 29619 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: No bootstrap required, opened a new log
I20260810 11:44:56.679227 29619 ts_tablet_manager.cc:1397] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 11:44:56.680280 29049 catalog_manager.cc:4841] TS c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201): tablet 6d5cc4edd01f4053805cb432553c5ba1 (table test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]) successfully deleted
I20260810 11:44:56.681288 29619 raft_consensus.cc:348] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.681381 29620 tablet_bootstrap.cc:492] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: No bootstrap required, opened a new log
I20260810 11:44:56.681488 29619 raft_consensus.cc:374] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.681545 29620 ts_tablet_manager.cc:1397] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260810 11:44:56.681608 29619 raft_consensus.cc:729] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de50fc4657448318ef93d556713c647, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.681886 29619 consensus_queue.cc:260] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.682426 29619 ts_tablet_manager.cc:1428] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:44:56.683017 29620 raft_consensus.cc:348] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.683262 29620 raft_consensus.cc:374] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.683391 29620 raft_consensus.cc:729] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.683784 29620 consensus_queue.cc:260] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.685806 29620 ts_tablet_manager.cc:1428] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:44:56.686048 29647 ts_tablet_manager.cc:1918] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:56.686254 29647 log.cc:1196] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/6d5cc4edd01f4053805cb432553c5ba1
I20260810 11:44:56.686863 29647 ts_tablet_manager.cc:1939] T 6d5cc4edd01f4053805cb432553c5ba1 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting consensus metadata
I20260810 11:44:56.688764 29049 catalog_manager.cc:4841] TS 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411): tablet 6d5cc4edd01f4053805cb432553c5ba1 (table test-table [id=ce6dc6dd27354f9a90a219a4507a0f7c]) successfully deleted
I20260810 11:44:56.877462 29630 raft_consensus.cc:482] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:56.877708 29630 raft_consensus.cc:504] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.878471 29630 leader_election.cc:288] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:56.879136 29237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6652e83662b64c28bc20fe295b944c71" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978" is_pre_election: true
I20260810 11:44:56.879256 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6652e83662b64c28bc20fe295b944c71" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647" is_pre_election: true
I20260810 11:44:56.879482 29237 raft_consensus.cc:2454] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:56.879613 29538 raft_consensus.cc:2454] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:56.879976 29308 leader_election.cc:302] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:56.880340 29630 raft_consensus.cc:2781] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:56.880488 29630 raft_consensus.cc:482] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:56.880605 29630 raft_consensus.cc:3037] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.883033 29630 raft_consensus.cc:504] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.883627 29630 leader_election.cc:288] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Requested vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:56.884166 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6652e83662b64c28bc20fe295b944c71" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647"
I20260810 11:44:56.884161 29237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6652e83662b64c28bc20fe295b944c71" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978"
I20260810 11:44:56.884392 29538 raft_consensus.cc:3037] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.884387 29237 raft_consensus.cc:3037] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:56.886677 29538 raft_consensus.cc:2454] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:56.887154 29305 leader_election.cc:302] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, 6de50fc4657448318ef93d556713c647; no voters:
I20260810 11:44:56.887475 29630 raft_consensus.cc:2781] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:56.887616 29237 raft_consensus.cc:2454] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:56.887646 29630 raft_consensus.cc:686] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Becoming Leader. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Running, Role: LEADER
I20260810 11:44:56.887933 29630 consensus_queue.cc:237] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.890990 29063 catalog_manager.cc:5495] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130). New cstate: current_term: 1 leader_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } health_report { overall_health: UNKNOWN } } }
W20260810 11:44:56.911300 29591 tablet.cc:2369] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:44:56 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362293, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.190@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:56.959436 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.944388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.20.93.190:38779)
0810 11:44:56.945349 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:56.945356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:56.945379 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:44:56.945443 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:56.945450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:56.945547 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:56.945732 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:56.946789 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.947998 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.949390 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.949879 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.950016 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.953738 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:56.953742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:56.953746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:56.953776 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:56.956407 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:56.957401 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:56.957405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:56.957407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:56.957522 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:56.957982 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:56.957985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:56.957987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:56.958264 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:56.958373 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:56.958945 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 11:44:56.959171 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":660,"threads_started":1}
I20260810 11:44:56.959860 29659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:56.944311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38779 (local address 127.0.0.1:42904)
0810 11:44:56.945002 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:56.945032 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:56.945207 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:56.945754 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:56.945757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:56.946409 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:44:56.946616 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.946626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.948150 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.948154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.949249 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:56.949261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:56.950037 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:56.950042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:56.950231 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:56.950887 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:44:56.950910 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:44:56.953487 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:44:56.956567 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:56.956583 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:56.956592 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:56.957283 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:56.957666 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:44:56.957674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:44:56.957679 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:44:56.957872 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:44:56.958424 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:44:56.958433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:44:56.958666 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 11:44:56.958880 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 11:44:56.959370 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":138,"thread_start_us":315,"threads_started":1}
I20260810 11:44:56.965699 29063 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:42904:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:44:56.967103 29063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:56.975073 29362 tablet_service.cc:1467] Processing CreateTablet for tablet 689c67b44cc541049a41efb730119bfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2582b1c66e9547f5a33a6c673786b188]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:56.975028 29210 tablet_service.cc:1467] Processing CreateTablet for tablet 689c67b44cc541049a41efb730119bfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2582b1c66e9547f5a33a6c673786b188]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:56.975659 29362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689c67b44cc541049a41efb730119bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.975690 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689c67b44cc541049a41efb730119bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.979638 29511 tablet_service.cc:1467] Processing CreateTablet for tablet 689c67b44cc541049a41efb730119bfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2582b1c66e9547f5a33a6c673786b188]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:44:56.980309 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689c67b44cc541049a41efb730119bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:56.982740 29620 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: Bootstrap starting.
I20260810 11:44:56.984687 29620 tablet_bootstrap.cc:654] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.985817 29621 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: Bootstrap starting.
I20260810 11:44:56.988065 29621 tablet_bootstrap.cc:654] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.988391 29619 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: Bootstrap starting.
I20260810 11:44:56.990237 29619 tablet_bootstrap.cc:654] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:56.995359 29619 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: No bootstrap required, opened a new log
I20260810 11:44:56.995362 29621 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: No bootstrap required, opened a new log
I20260810 11:44:56.995548 29621 ts_tablet_manager.cc:1397] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260810 11:44:56.995548 29619 ts_tablet_manager.cc:1397] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260810 11:44:56.996309 29621 raft_consensus.cc:348] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.996315 29619 raft_consensus.cc:348] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.996574 29621 raft_consensus.cc:374] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.996575 29619 raft_consensus.cc:374] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:56.996659 29619 raft_consensus.cc:729] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de50fc4657448318ef93d556713c647, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.996665 29621 raft_consensus.cc:729] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a74dbb3baa412782f6a07bdbf77978, State: Initialized, Role: FOLLOWER
I20260810 11:44:56.996928 29619 consensus_queue.cc:260] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.996917 29621 consensus_queue.cc:260] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:56.997417 29627 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:56.997488 29628 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:56.997891 29628 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:56.997889 29627 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:56.998363 29621 ts_tablet_manager.cc:1428] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 11:44:56.998399 29619 ts_tablet_manager.cc:1428] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 11:44:56.999470 29620 tablet_bootstrap.cc:492] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: No bootstrap required, opened a new log
I20260810 11:44:56.999747 29620 ts_tablet_manager.cc:1397] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
I20260810 11:44:57.000566 29620 raft_consensus.cc:348] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.000914 29620 raft_consensus.cc:374] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.001068 29620 raft_consensus.cc:729] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.001446 29620 consensus_queue.cc:260] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.002226 29620 ts_tablet_manager.cc:1428] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 11:44:57.002018 29630 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.002491 29630 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:57.284930 29628 raft_consensus.cc:482] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:57.285133 29628 raft_consensus.cc:504] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.286739 29628 leader_election.cc:288] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411)
I20260810 11:44:57.294641 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.287183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:57827 (local address 127.20.93.129:35201)
0810 11:44:57.287698 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.287706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.287777 (+ 71us) server_negotiation.cc:407] Connection header received
0810 11:44:57.287858 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.287863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.287945 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.288847 (+ 902us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:57.289122 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.290624 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.292573 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.293526 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.293772 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.293855 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.294153 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.294304 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
I20260810 11:44:57.294695 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.286999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:35201 (local address 127.20.93.131:57827)
0810 11:44:57.287517 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:57.287538 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:57.287678 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:57.288157 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:57.288166 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:57.288207 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:57.288628 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.288644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.290800 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.290804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.292410 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.292421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.293771 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.293777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.293980 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.294006 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 11:44:57.294123 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 11:44:57.294269 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":1,"thread_start_us":291,"threads_started":1}
I20260810 11:44:57.295331 29237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689c67b44cc541049a41efb730119bfe" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978" is_pre_election: true
I20260810 11:44:57.295591 29237 raft_consensus.cc:2454] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 0.
I20260810 11:44:57.296123 29461 leader_election.cc:302] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de50fc4657448318ef93d556713c647, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.296545 29628 raft_consensus.cc:2781] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:57.296692 29628 raft_consensus.cc:482] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:57.296785 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.287863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:56299 (local address 127.20.93.130:39411)
0810 11:44:57.291164 (+ 3301us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.291169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.291185 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:44:57.291243 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.291247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.291334 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.291416 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:44:57.292045 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.292896 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.294579 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.295762 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.295944 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.296026 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.296328 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.296470 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3197,"thread_start_us":3088,"threads_started":1}
I20260810 11:44:57.296814 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.287736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:39411 (local address 127.20.93.131:56299)
0810 11:44:57.288244 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:57.288258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:57.288354 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:57.291483 (+ 3129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:57.291489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:57.291518 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:44:57.291866 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.291877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.293026 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.293030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.294392 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.294408 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.295965 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.295973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.296153 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.296181 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 11:44:57.296296 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:44:57.296427 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":238,"threads_started":1}
I20260810 11:44:57.296829 29628 raft_consensus.cc:3037] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.297677 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689c67b44cc541049a41efb730119bfe" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" is_pre_election: true
I20260810 11:44:57.298101 29384 raft_consensus.cc:2454] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 0.
I20260810 11:44:57.299757 29628 raft_consensus.cc:504] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.300419 29628 leader_election.cc:288] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 election: Requested vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411)
I20260810 11:44:57.300896 29237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689c67b44cc541049a41efb730119bfe" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978"
I20260810 11:44:57.300940 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689c67b44cc541049a41efb730119bfe" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64ff25bf5e42e7837102f0c95a55f8"
I20260810 11:44:57.301119 29237 raft_consensus.cc:3037] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.301198 29384 raft_consensus.cc:3037] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.303273 29237 raft_consensus.cc:2454] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 1.
I20260810 11:44:57.303436 29384 raft_consensus.cc:2454] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 1.
I20260810 11:44:57.303718 29461 leader_election.cc:302] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de50fc4657448318ef93d556713c647, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.304049 29628 raft_consensus.cc:2781] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:57.304558 29628 raft_consensus.cc:686] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [term 1 LEADER]: Becoming Leader. State: Replica: 6de50fc4657448318ef93d556713c647, State: Running, Role: LEADER
I20260810 11:44:57.305023 29628 consensus_queue.cc:237] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.307106 29669 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: TxnStatusTablet state changed. Reason: New leader 6de50fc4657448318ef93d556713c647. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.307525 29669 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:57.308295 29671 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:44:57.308519 29671 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:44:57.308830 29063 catalog_manager.cc:5495] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6de50fc4657448318ef93d556713c647 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } health_report { overall_health: HEALTHY } } }
I20260810 11:44:57.352240 29630 consensus_queue.cc:1035] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.359903 29633 consensus_queue.cc:1035] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.375808 29659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.366697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:32989 (local address 127.0.0.1:38734)
0810 11:44:57.366906 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:57.366930 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:57.367072 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:57.370509 (+ 3437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:57.370517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:57.370536 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:57.370756 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.370769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.372016 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.372023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.373062 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.373074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.373927 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.373933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.374129 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.374190 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.375185 (+ 995us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.375212 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:44:57.375295 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 11:44:57.375460 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 11:44:57.375844 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.366849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.20.93.131:32989)
0810 11:44:57.370084 (+ 3235us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.370088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.370113 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:44:57.370193 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.370197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.370288 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.370470 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:57.370962 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.371872 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.373208 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.373769 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.373902 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.374426 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.375040 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.375251 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.375329 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.375419 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3086,"thread_start_us":2918,"threads_started":1}
I20260810 11:44:57.380457 29237 raft_consensus.cc:1261] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Refusing update from remote peer 6de50fc4657448318ef93d556713c647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.380928 29384 raft_consensus.cc:1261] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Refusing update from remote peer 6de50fc4657448318ef93d556713c647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.381826 29669 consensus_queue.cc:1035] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.382181 29628 consensus_queue.cc:1035] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.393019 29630 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: TxnStatusTablet state changed. Reason: New leader 6de50fc4657448318ef93d556713c647. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.393317 29630 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:57.402393 29627 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: TxnStatusTablet state changed. Reason: New leader 6de50fc4657448318ef93d556713c647. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.402726 29627 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:57.406160 29628 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.406498 29628 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:57.408535 29630 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.408777 29630 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 3a64ff25bf5e42e7837102f0c95a55f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:57.411634 29628 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.411980 29628 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P 6de50fc4657448318ef93d556713c647: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:44:57.417932 29627 tablet_replica.cc:439] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } } }
I20260810 11:44:57.418177 29627 tablet_replica.cc:442] T 689c67b44cc541049a41efb730119bfe P c0a74dbb3baa412782f6a07bdbf77978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:44:57.437436 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.431913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:36199 (local address 127.20.93.131:32989)
0810 11:44:57.432147 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.432153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.432462 (+ 309us) server_negotiation.cc:407] Connection header received
0810 11:44:57.432677 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.432687 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.432756 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.432850 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:57.433304 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.434137 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.435547 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.436094 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.436240 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.436648 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.436760 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.436867 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.437002 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.437083 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:44:57.437530 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.431785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:32989 (local address 127.20.93.130:36199)
0810 11:44:57.432358 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:57.432374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:57.432539 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:57.432867 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:57.432871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:57.432884 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:57.433154 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.433163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.434290 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.434296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.435401 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.435413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.436249 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.436254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.436403 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.436460 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.436867 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.436881 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 11:44:57.436980 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 11:44:57.437119 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":277,"threads_started":1}
I20260810 11:44:57.479837 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.472452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:41649 (local address 127.20.93.130:39411)
0810 11:44:57.472895 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.472903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.473406 (+ 503us) server_negotiation.cc:407] Connection header received
0810 11:44:57.473880 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.473886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.473977 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.474079 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:57.474932 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.475663 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.477930 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.478474 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.478575 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.479197 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.479308 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.479377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.479539 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.479609 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260810 11:44:57.480080 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.472513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:39411 (local address 127.20.93.131:41649)
0810 11:44:57.473281 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 11:44:57.473310 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:44:57.473727 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:44:57.474250 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:44:57.474256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:44:57.474269 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:44:57.474748 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.474762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.475808 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.475814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.477744 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:44:57.477760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.478668 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.478676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:44:57.478872 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.478949 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.479398 (+ 449us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.479417 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 11:44:57.479496 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 11:44:57.479648 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":409,"threads_started":1}
I20260810 11:44:57.513819 29061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.514065 29061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.517071 29061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Sending DeleteTablet for 3 replicas of tablet 6652e83662b64c28bc20fe295b944c71
I20260810 11:44:57.517854 29210 tablet_service.cc:1514] Processing DeleteTablet for tablet 6652e83662b64c28bc20fe295b944c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43996
I20260810 11:44:57.517936 29362 tablet_service.cc:1514] Processing DeleteTablet for tablet 6652e83662b64c28bc20fe295b944c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38838
I20260810 11:44:57.518368 29511 tablet_service.cc:1514] Processing DeleteTablet for tablet 6652e83662b64c28bc20fe295b944c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260810 11:44:57.530109 29699 tablet_replica.cc:330] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: stopping tablet replica
I20260810 11:44:57.531044 29699 raft_consensus.cc:2229] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.531788 29699 raft_consensus.cc:2258] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.534228 29700 tablet_replica.cc:330] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: stopping tablet replica
I20260810 11:44:57.534458 29700 raft_consensus.cc:2229] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:57.534614 29701 tablet_replica.cc:330] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: stopping tablet replica
I20260810 11:44:57.534845 29701 raft_consensus.cc:2229] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.534871 29700 raft_consensus.cc:2258] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.535144 29701 raft_consensus.cc:2258] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.535853 29699 ts_tablet_manager.cc:1905] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.536204 29701 ts_tablet_manager.cc:1905] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.539572 29700 ts_tablet_manager.cc:1905] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.543708 29699 ts_tablet_manager.cc:1918] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:57.543937 29701 ts_tablet_manager.cc:1918] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:57.544245 29701 log.cc:1196] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/6652e83662b64c28bc20fe295b944c71
I20260810 11:44:57.544888 29701 ts_tablet_manager.cc:1939] T 6652e83662b64c28bc20fe295b944c71 P 6de50fc4657448318ef93d556713c647: Deleting consensus metadata
I20260810 11:44:57.546355 29046 catalog_manager.cc:4841] TS 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989): tablet 6652e83662b64c28bc20fe295b944c71 (table test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]) successfully deleted
I20260810 11:44:57.548375 29700 ts_tablet_manager.cc:1918] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:57.548537 29700 log.cc:1196] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/6652e83662b64c28bc20fe295b944c71
I20260810 11:44:57.548956 29700 ts_tablet_manager.cc:1939] T 6652e83662b64c28bc20fe295b944c71 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting consensus metadata
I20260810 11:44:57.549786 29699 log.cc:1196] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/6652e83662b64c28bc20fe295b944c71
I20260810 11:44:57.550112 29049 catalog_manager.cc:4841] TS 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411): tablet 6652e83662b64c28bc20fe295b944c71 (table test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]) successfully deleted
I20260810 11:44:57.550824 29699 ts_tablet_manager.cc:1939] T 6652e83662b64c28bc20fe295b944c71 P c0a74dbb3baa412782f6a07bdbf77978: Deleting consensus metadata
I20260810 11:44:57.551810 29049 catalog_manager.cc:4841] TS c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201): tablet 6652e83662b64c28bc20fe295b944c71 (table test-table [id=af9054d41f8b4a0aa1b88926ee58ef09]) successfully deleted
Aug 10 11:44:57 dist-test-slave-njxd krb5kdc[29014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:44:57 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:44:57 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:44:57.607599 29709 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:44:57.596087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.20.93.190:38779)
0810 11:44:57.597034 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.597038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.597055 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:44:57.597110 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.597114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.597191 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.597322 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:57.598796 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.599589 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.600864 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.601352 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.601495 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.607262 (+ 5767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42918: BlockingRecv error: recv got EOF from 127.0.0.1:42918 (error 108)
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":607,"threads_started":1}
Aug 10 11:44:57 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM
I20260810 11:44:57.625926 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.611038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.20.93.190:38779)
0810 11:44:57.611215 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.611221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.611249 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:44:57.611354 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.611359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.611432 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.611538 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:57.612910 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.613936 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.615381 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.615805 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.615942 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.619870 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:57.619875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:57.619878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:57.619914 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:57.623101 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:57.623811 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:57.623819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:57.623822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:57.623975 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:57.624359 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:57.624365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:57.624368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:57.624668 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:57.624951 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.625268 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.625506 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 11:44:57.630769 29061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:57.631479 29061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:57.638859 29210 tablet_service.cc:1467] Processing CreateTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 (DEFAULT_TABLE table=test-table [id=b23d31d9215b42848e941deb0ea7f184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.639236 29362 tablet_service.cc:1467] Processing CreateTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 (DEFAULT_TABLE table=test-table [id=b23d31d9215b42848e941deb0ea7f184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.639401 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ba7aab71546c19f3e1d2e1326a363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.639355 29511 tablet_service.cc:1467] Processing CreateTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 (DEFAULT_TABLE table=test-table [id=b23d31d9215b42848e941deb0ea7f184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.639634 29362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ba7aab71546c19f3e1d2e1326a363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.639875 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00ba7aab71546c19f3e1d2e1326a363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.654959 29717 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Bootstrap starting.
I20260810 11:44:57.656145 29716 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Bootstrap starting.
I20260810 11:44:57.658084 29719 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Bootstrap starting.
I20260810 11:44:57.658299 29716 tablet_bootstrap.cc:654] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.658785 29717 tablet_bootstrap.cc:654] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.660274 29719 tablet_bootstrap.cc:654] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.667287 29716 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: No bootstrap required, opened a new log
I20260810 11:44:57.667484 29716 ts_tablet_manager.cc:1397] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260810 11:44:57.668131 29716 raft_consensus.cc:348] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.668397 29716 raft_consensus.cc:374] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.668506 29716 raft_consensus.cc:729] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a74dbb3baa412782f6a07bdbf77978, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.668784 29716 consensus_queue.cc:260] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.669433 29716 ts_tablet_manager.cc:1428] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 11:44:57.673281 29717 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: No bootstrap required, opened a new log
I20260810 11:44:57.673552 29717 ts_tablet_manager.cc:1397] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.005s
I20260810 11:44:57.674357 29717 raft_consensus.cc:348] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.674595 29717 raft_consensus.cc:374] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.674698 29717 raft_consensus.cc:729] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.674971 29717 consensus_queue.cc:260] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.675580 29717 ts_tablet_manager.cc:1428] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 11:44:57.679167 29719 tablet_bootstrap.cc:492] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: No bootstrap required, opened a new log
I20260810 11:44:57.679373 29719 ts_tablet_manager.cc:1397] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.006s
I20260810 11:44:57.680289 29719 raft_consensus.cc:348] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.680565 29719 raft_consensus.cc:374] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.680676 29719 raft_consensus.cc:729] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de50fc4657448318ef93d556713c647, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.680953 29719 consensus_queue.cc:260] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.681656 29719 ts_tablet_manager.cc:1428] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 11:44:57.701332 29669 raft_consensus.cc:482] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:57.701598 29669 raft_consensus.cc:504] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.702322 29669 leader_election.cc:288] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411)
I20260810 11:44:57.702905 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ba7aab71546c19f3e1d2e1326a363" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978" is_pre_election: true
I20260810 11:44:57.703006 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ba7aab71546c19f3e1d2e1326a363" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" is_pre_election: true
I20260810 11:44:57.703151 29238 raft_consensus.cc:2454] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 0.
I20260810 11:44:57.703310 29384 raft_consensus.cc:2454] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 0.
I20260810 11:44:57.703660 29461 leader_election.cc:302] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de50fc4657448318ef93d556713c647, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.704005 29669 raft_consensus.cc:2781] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:57.704124 29669 raft_consensus.cc:482] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:57.704250 29669 raft_consensus.cc:3037] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.706931 29669 raft_consensus.cc:504] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.707602 29669 leader_election.cc:288] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 election: Requested vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411)
I20260810 11:44:57.708194 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ba7aab71546c19f3e1d2e1326a363" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978"
I20260810 11:44:57.708315 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00ba7aab71546c19f3e1d2e1326a363" candidate_uuid: "6de50fc4657448318ef93d556713c647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64ff25bf5e42e7837102f0c95a55f8"
I20260810 11:44:57.708442 29238 raft_consensus.cc:3037] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.708572 29384 raft_consensus.cc:3037] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.710697 29238 raft_consensus.cc:2454] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 1.
I20260810 11:44:57.711156 29461 leader_election.cc:302] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de50fc4657448318ef93d556713c647, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.711756 29669 raft_consensus.cc:2781] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:57.712287 29669 raft_consensus.cc:686] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 1 LEADER]: Becoming Leader. State: Replica: 6de50fc4657448318ef93d556713c647, State: Running, Role: LEADER
I20260810 11:44:57.712512 29384 raft_consensus.cc:2454] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de50fc4657448318ef93d556713c647 in term 1.
I20260810 11:44:57.712699 29669 consensus_queue.cc:237] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.716153 29063 catalog_manager.cc:5495] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6de50fc4657448318ef93d556713c647 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "6de50fc4657448318ef93d556713c647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } health_report { overall_health: HEALTHY } } }
I20260810 11:44:57.741087 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.735461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.20.93.131:32989)
0810 11:44:57.735663 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.735669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.735694 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:44:57.735956 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.735961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.736039 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.736161 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:57.736563 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.737373 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.739029 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.739606 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.739773 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.740291 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.740418 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.740519 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.740647 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.740733 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:44:57.744987 29238 raft_consensus.cc:1261] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Refusing update from remote peer 6de50fc4657448318ef93d556713c647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.745114 29384 raft_consensus.cc:1261] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Refusing update from remote peer 6de50fc4657448318ef93d556713c647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.745774 29669 consensus_queue.cc:1035] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.746088 29628 consensus_queue.cc:1035] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.764631 29063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.764961 29063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.768288 29063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Sending DeleteTablet for 3 replicas of tablet f00ba7aab71546c19f3e1d2e1326a363
I20260810 11:44:57.769338 29210 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43996
I20260810 11:44:57.769318 29362 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38838
I20260810 11:44:57.769430 29511 tablet_service.cc:1514] Processing DeleteTablet for tablet f00ba7aab71546c19f3e1d2e1326a363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260810 11:44:57.769665 29700 tablet_replica.cc:330] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: stopping tablet replica
I20260810 11:44:57.769665 29699 tablet_replica.cc:330] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: stopping tablet replica
I20260810 11:44:57.769872 29700 raft_consensus.cc:2229] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.769872 29699 raft_consensus.cc:2229] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.770103 29700 raft_consensus.cc:2258] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.770103 29699 raft_consensus.cc:2258] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.770398 29701 tablet_replica.cc:330] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: stopping tablet replica
I20260810 11:44:57.770658 29701 raft_consensus.cc:2229] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:57.771268 29701 raft_consensus.cc:2258] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.771566 29700 ts_tablet_manager.cc:1905] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.771642 29063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:44:57.772487 29063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:44:57.772713 29701 ts_tablet_manager.cc:1905] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.780821 29701 ts_tablet_manager.cc:1918] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:57.781040 29701 log.cc:1196] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/f00ba7aab71546c19f3e1d2e1326a363
I20260810 11:44:57.781216 29210 tablet_service.cc:1467] Processing CreateTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 (DEFAULT_TABLE table=test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.781615 29511 tablet_service.cc:1467] Processing CreateTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 (DEFAULT_TABLE table=test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.782040 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce24bbe90fb040789c46b2c8a570b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.782150 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce24bbe90fb040789c46b2c8a570b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.782510 29362 tablet_service.cc:1467] Processing CreateTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 (DEFAULT_TABLE table=test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:44:57.783010 29362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce24bbe90fb040789c46b2c8a570b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:44:57.788920 29716 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Bootstrap starting.
I20260810 11:44:57.790717 29716 tablet_bootstrap.cc:654] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.791160 29700 ts_tablet_manager.cc:1918] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:57.791365 29700 log.cc:1196] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/f00ba7aab71546c19f3e1d2e1326a363
I20260810 11:44:57.791816 29700 ts_tablet_manager.cc:1939] T f00ba7aab71546c19f3e1d2e1326a363 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting consensus metadata
I20260810 11:44:57.781620 29701 ts_tablet_manager.cc:1939] T f00ba7aab71546c19f3e1d2e1326a363 P 6de50fc4657448318ef93d556713c647: Deleting consensus metadata
I20260810 11:44:57.792917 29719 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Bootstrap starting.
I20260810 11:44:57.793833 29717 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Bootstrap starting.
I20260810 11:44:57.795127 29716 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: No bootstrap required, opened a new log
I20260810 11:44:57.795315 29716 ts_tablet_manager.cc:1397] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260810 11:44:57.795601 29717 tablet_bootstrap.cc:654] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.796202 29716 raft_consensus.cc:348] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.796490 29716 raft_consensus.cc:374] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.796588 29716 raft_consensus.cc:729] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a74dbb3baa412782f6a07bdbf77978, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.796861 29719 tablet_bootstrap.cc:654] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Neither blocks nor log segments found. Creating new log.
I20260810 11:44:57.797005 29046 catalog_manager.cc:4841] TS 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989): tablet f00ba7aab71546c19f3e1d2e1326a363 (table test-table [id=b23d31d9215b42848e941deb0ea7f184]) successfully deleted
I20260810 11:44:57.796916 29716 consensus_queue.cc:260] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.798352 29716 ts_tablet_manager.cc:1428] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 11:44:57.800252 29719 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: No bootstrap required, opened a new log
I20260810 11:44:57.800474 29719 ts_tablet_manager.cc:1397] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260810 11:44:57.801295 29719 raft_consensus.cc:348] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.801607 29719 raft_consensus.cc:374] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.801698 29719 raft_consensus.cc:729] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de50fc4657448318ef93d556713c647, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.801980 29719 consensus_queue.cc:260] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.804102 29699 ts_tablet_manager.cc:1905] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.804538 29719 ts_tablet_manager.cc:1428] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 11:44:57.809875 29699 ts_tablet_manager.cc:1918] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:44:57.810052 29699 log.cc:1196] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/f00ba7aab71546c19f3e1d2e1326a363
I20260810 11:44:57.810459 29699 ts_tablet_manager.cc:1939] T f00ba7aab71546c19f3e1d2e1326a363 P c0a74dbb3baa412782f6a07bdbf77978: Deleting consensus metadata
I20260810 11:44:57.811439 29049 catalog_manager.cc:4841] TS c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201): tablet f00ba7aab71546c19f3e1d2e1326a363 (table test-table [id=b23d31d9215b42848e941deb0ea7f184]) successfully deleted
I20260810 11:44:57.811830 29049 catalog_manager.cc:4841] TS 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411): tablet f00ba7aab71546c19f3e1d2e1326a363 (table test-table [id=b23d31d9215b42848e941deb0ea7f184]) successfully deleted
I20260810 11:44:57.813179 29717 tablet_bootstrap.cc:492] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: No bootstrap required, opened a new log
I20260810 11:44:57.813382 29717 ts_tablet_manager.cc:1397] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.007s
I20260810 11:44:57.814276 29717 raft_consensus.cc:348] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.814577 29717 raft_consensus.cc:374] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:44:57.814675 29717 raft_consensus.cc:729] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Initialized, Role: FOLLOWER
I20260810 11:44:57.814929 29717 consensus_queue.cc:260] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.815486 29717 ts_tablet_manager.cc:1428] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 11:44:57.852351 29630 raft_consensus.cc:482] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:44:57.852571 29630 raft_consensus.cc:504] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.853302 29630 leader_election.cc:288] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:57.853927 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce24bbe90fb040789c46b2c8a570b0f3" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978" is_pre_election: true
I20260810 11:44:57.853978 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce24bbe90fb040789c46b2c8a570b0f3" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647" is_pre_election: true
I20260810 11:44:57.854208 29238 raft_consensus.cc:2454] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:57.854255 29538 raft_consensus.cc:2454] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 0.
I20260810 11:44:57.854704 29308 leader_election.cc:302] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.855011 29630 raft_consensus.cc:2781] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:44:57.855134 29630 raft_consensus.cc:482] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:44:57.855266 29630 raft_consensus.cc:3037] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.857450 29630 raft_consensus.cc:504] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.858212 29630 leader_election.cc:288] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Requested vote from peers c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201), 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989)
I20260810 11:44:57.858731 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce24bbe90fb040789c46b2c8a570b0f3" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a74dbb3baa412782f6a07bdbf77978"
I20260810 11:44:57.858968 29238 raft_consensus.cc:3037] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.859150 29538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce24bbe90fb040789c46b2c8a570b0f3" candidate_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de50fc4657448318ef93d556713c647"
I20260810 11:44:57.859395 29538 raft_consensus.cc:3037] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:44:57.862082 29238 raft_consensus.cc:2454] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:57.862556 29308 leader_election.cc:302] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64ff25bf5e42e7837102f0c95a55f8, c0a74dbb3baa412782f6a07bdbf77978; no voters:
I20260810 11:44:57.862720 29538 raft_consensus.cc:2454] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a64ff25bf5e42e7837102f0c95a55f8 in term 1.
I20260810 11:44:57.862924 29630 raft_consensus.cc:2781] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:44:57.863137 29630 raft_consensus.cc:686] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Becoming Leader. State: Replica: 3a64ff25bf5e42e7837102f0c95a55f8, State: Running, Role: LEADER
I20260810 11:44:57.863476 29630 consensus_queue.cc:237] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } }
I20260810 11:44:57.866575 29061 catalog_manager.cc:5495] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130). New cstate: current_term: 1 leader_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a64ff25bf5e42e7837102f0c95a55f8" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 39411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 } health_report { overall_health: UNKNOWN } } }
I20260810 11:44:57.896950 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.891994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.20.93.130:39411)
0810 11:44:57.892507 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:57.892511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:57.892525 (+ 14us) server_negotiation.cc:407] Connection header received
0810 11:44:57.892573 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:57.892577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:57.892649 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:57.892756 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:44:57.893131 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.893967 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.895283 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:57.895703 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:57.895843 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:57.896255 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:44:57.896353 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:44:57.896448 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:57.896578 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 11:44:57.896666 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409}
I20260810 11:44:57.906612 29538 raft_consensus.cc:1261] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Refusing update from remote peer 3a64ff25bf5e42e7837102f0c95a55f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.906672 29238 raft_consensus.cc:1261] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Refusing update from remote peer 3a64ff25bf5e42e7837102f0c95a55f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:44:57.907217 29630 consensus_queue.cc:1035] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a74dbb3baa412782f6a07bdbf77978" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.907577 29633 consensus_queue.cc:1035] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de50fc4657448318ef93d556713c647" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:44:57.977510 29061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.977789 29061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:44:57.980834 29061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 393d2ded76084279bee7f41a4e45b849: Sending DeleteTablet for 3 replicas of tablet ce24bbe90fb040789c46b2c8a570b0f3
I20260810 11:44:57.981602 29210 tablet_service.cc:1514] Processing DeleteTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43996
I20260810 11:44:57.981925 29699 tablet_replica.cc:330] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: stopping tablet replica
I20260810 11:44:57.981887 29362 tablet_service.cc:1514] Processing DeleteTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38838
I20260810 11:44:57.982113 29699 raft_consensus.cc:2229] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.982141 29700 tablet_replica.cc:330] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: stopping tablet replica
I20260810 11:44:57.982316 29700 raft_consensus.cc:2229] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:44:57.982364 29699 raft_consensus.cc:2258] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.982678 29700 raft_consensus.cc:2258] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.983520 29699 ts_tablet_manager.cc:1905] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.983701 29700 ts_tablet_manager.cc:1905] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.990100 29511 tablet_service.cc:1514] Processing DeleteTablet for tablet ce24bbe90fb040789c46b2c8a570b0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:44:57 UTC) from {username='oryx', principal='oryx/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260810 11:44:57.990481 29701 tablet_replica.cc:330] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: stopping tablet replica
I20260810 11:44:57.990454 29700 ts_tablet_manager.cc:1918] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:57.990587 29700 log.cc:1196] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/ce24bbe90fb040789c46b2c8a570b0f3
I20260810 11:44:57.990688 29701 raft_consensus.cc:2229] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:44:57.990921 29701 raft_consensus.cc:2258] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:44:57.991061 29700 ts_tablet_manager.cc:1939] T ce24bbe90fb040789c46b2c8a570b0f3 P 3a64ff25bf5e42e7837102f0c95a55f8: Deleting consensus metadata
I20260810 11:44:57.991484 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29138
I20260810 11:44:57.991860 29701 ts_tablet_manager.cc:1905] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:44:57.992497 29049 catalog_manager.cc:4841] TS 3a64ff25bf5e42e7837102f0c95a55f8 (127.20.93.130:39411): tablet ce24bbe90fb040789c46b2c8a570b0f3 (table test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]) successfully deleted
I20260810 11:44:58.002331 29701 ts_tablet_manager.cc:1918] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:58.002534 29701 log.cc:1196] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/ce24bbe90fb040789c46b2c8a570b0f3
I20260810 11:44:58.003105 29701 ts_tablet_manager.cc:1939] T ce24bbe90fb040789c46b2c8a570b0f3 P 6de50fc4657448318ef93d556713c647: Deleting consensus metadata
I20260810 11:44:58.004577 29046 catalog_manager.cc:4841] TS 6de50fc4657448318ef93d556713c647 (127.20.93.131:32989): tablet ce24bbe90fb040789c46b2c8a570b0f3 (table test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]) successfully deleted
I20260810 11:44:58.005267 29699 ts_tablet_manager.cc:1918] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:44:58.005474 29699 log.cc:1196] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/ce24bbe90fb040789c46b2c8a570b0f3
I20260810 11:44:58.006060 29699 ts_tablet_manager.cc:1939] T ce24bbe90fb040789c46b2c8a570b0f3 P c0a74dbb3baa412782f6a07bdbf77978: Deleting consensus metadata
I20260810 11:44:58.007453 29049 catalog_manager.cc:4841] TS c0a74dbb3baa412782f6a07bdbf77978 (127.20.93.129:35201): tablet ce24bbe90fb040789c46b2c8a570b0f3 (table test-table [id=0a102e6ed74e4c009a1d74abe2a2577a]) successfully deleted
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.129@KRBTEST.COM
I20260810 11:44:58.024673 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:57.993020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44010 (local address 127.20.93.129:35201)
0810 11:44:58.008199 (+ 15179us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:58.008206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:58.008335 (+ 129us) server_negotiation.cc:407] Connection header received
0810 11:44:58.008514 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:58.008520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:58.008611 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:58.008730 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:58.010027 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.011198 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.012452 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.013179 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.013311 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:58.017567 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:58.017611 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:58.017633 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:58.017684 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:58.021393 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.022361 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.022372 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.022376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.022551 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.022948 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.022953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.022957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.023490 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:58.023843 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:58.024008 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 11:44:58.024275 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15017,"thread_start_us":15201,"threads_started":1}
I20260810 11:44:58.025477 29280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:58.130568 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29138
I20260810 11:44:58.159745 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29289
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.130@KRBTEST.COM
I20260810 11:44:58.180752 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:58.160547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.20.93.130:39411)
0810 11:44:58.160749 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:58.160757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:58.160815 (+ 58us) server_negotiation.cc:407] Connection header received
0810 11:44:58.161047 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:58.161053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:58.161159 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:58.161277 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:58.162981 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.164159 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.166239 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.166876 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.167023 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:58.172972 (+ 5949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:58.173000 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:58.173027 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:58.173086 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:58.177172 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.178262 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.178273 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.178277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.178454 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.178931 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.178936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.178939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.179419 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:58.179727 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:58.180147 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 11:44:58.180406 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:44:58.181501 29432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:58.313803 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29289
I20260810 11:44:58.338569 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29441
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.93.131@KRBTEST.COM
I20260810 11:44:58.354883 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:44:58.339378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.20.93.131:32989)
0810 11:44:58.340254 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0810 11:44:58.340259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:44:58.340274 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:44:58.340331 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:44:58.340334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:44:58.340403 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:44:58.340495 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:44:58.341883 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.342718 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.344168 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:44:58.344585 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:44:58.344698 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:44:58.349009 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:44:58.349026 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:44:58.349045 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:44:58.349078 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:44:58.352042 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.352683 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.352690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.352692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.352857 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:44:58.353180 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:44:58.353183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:44:58.353186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:44:58.353527 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:44:58.353823 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 11:44:58.354300 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 11:44:58.354475 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":636,"threads_started":1}
I20260810 11:44:58.355592 29585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:58.503682 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29441
I20260810 11:44:58.533401 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29027
I20260810 11:44:58.534624 29106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:44:58.653113 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29027
2026-08-10T11:44:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7153 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29749](info): set up 2 sockets
Aug 10 11:44:58 dist-test-slave-njxd krb5kdc[29749](info): commencing operation
krb5kdc: starting...
W20260810 11:44:59.832010 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:45:00 dist-test-slave-njxd krb5kdc[29749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:45:00.139784 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40297
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:40297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:00.497366 29764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:00.497816 29764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:00.497983 29764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:00.505745 29764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:00.505896 29764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:00.506000 29764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:00.506102 29764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:00.506196 29764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:00.516865 29764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:40297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:00.518370 29764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:00.519757 29764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:00.532783 29769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:00.532742 29770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:00.532761 29772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:00.533841 29764 server_base.cc:1034] running on GCE node
I20260810 11:45:00.534747 29764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:00.536389 29764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:00.537693 29764 hybrid_clock.cc:648] HybridClock initialized: now 1786362300537599 us; error 63 us; skew 500 ppm
I20260810 11:45:00.538061 29764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T11:45:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1948 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:45:00 dist-test-slave-njxd krb5kdc[29778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:45:00 dist-test-slave-njxd krb5kdc[29778](info): set up 2 sockets
Aug 10 11:45:00 dist-test-slave-njxd krb5kdc[29778](info): commencing operation
krb5kdc: starting...
W20260810 11:45:01.789171 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:45:01 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:45:02.070268 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:46325
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:46325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:02.418533 29792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:02.418848 29792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:02.418939 29792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:02.426218 29792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:02.426342 29792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:02.426388 29792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:02.426438 29792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:02.426482 29792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:02.436365 29792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:46325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:46325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:02.437675 29792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:02.439317 29792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:02.451820 29797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:02.451819 29800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:02.451859 29798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:02.452941 29792 server_base.cc:1034] running on GCE node
I20260810 11:45:02.453790 29792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:02.455253 29792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:02.456493 29792 hybrid_clock.cc:648] HybridClock initialized: now 1786362302456448 us; error 36 us; skew 500 ppm
I20260810 11:45:02.456845 29792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:02 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362302, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:02.459993 29792 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:45:02.461500 29792 webserver.cc:460] Webserver started at http://127.20.93.190:42551/ using document root <none> and password file <none>
I20260810 11:45:02.462234 29792 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:02.462383 29792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:02.462715 29792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:02.465816 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "5a58cd892e2246c9a8d86338b9725210"
format_stamp: "Formatted at 2026-08-10 11:45:02 on dist-test-slave-njxd"
server_key: "ec26490a26a49204461943f1e5bf549b"
server_key_iv: "78c82ebf422a77f469af1dbd3a0f0d4d"
server_key_version: "encryptionkey@0"
I20260810 11:45:02.467028 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "5a58cd892e2246c9a8d86338b9725210"
format_stamp: "Formatted at 2026-08-10 11:45:02 on dist-test-slave-njxd"
server_key: "ec26490a26a49204461943f1e5bf549b"
server_key_iv: "78c82ebf422a77f469af1dbd3a0f0d4d"
server_key_version: "encryptionkey@0"
I20260810 11:45:02.472389 29792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:45:02.475943 29808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:02.477329 29792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:45:02.477528 29792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "5a58cd892e2246c9a8d86338b9725210"
format_stamp: "Formatted at 2026-08-10 11:45:02 on dist-test-slave-njxd"
server_key: "ec26490a26a49204461943f1e5bf549b"
server_key_iv: "78c82ebf422a77f469af1dbd3a0f0d4d"
server_key_version: "encryptionkey@0"
I20260810 11:45:02.477780 29792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:02.507489 29792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:02.511834 29792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:02.512144 29792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:02.531277 29792 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:46325
I20260810 11:45:02.531302 29862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:46325 every 8 connection(s)
I20260810 11:45:02.532733 29792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:45:02.536608 29863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:02.540094 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29792
I20260810 11:45:02.540326 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:45:02.541041 20854 external_mini_cluster.cc:1428] Setting key c60c63200c8eb82e6c3369dbcf957eb1
I20260810 11:45:02.549399 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Bootstrap starting.
I20260810 11:45:02.553144 29863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:02.554291 29863 log.cc:824] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:45:02 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:02.557308 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: No bootstrap required, opened a new log
I20260810 11:45:02.564364 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:02.543180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.20.93.190:46325)
0810 11:45:02.544083 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:02.544103 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:02.544151 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:45:02.545888 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:02.545913 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:02.546353 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:02.547052 (+ 699us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:02.548619 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:02.550015 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:02.551322 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:02.552032 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:02.552251 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:02.556560 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:02.556567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:02.556588 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:02.556641 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:02.559920 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:02.560665 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:02.560672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:02.560679 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:02.560860 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:02.561294 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:02.561299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:02.561304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:02.561941 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:02.562286 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:02.562761 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 11:45:02.563246 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":375,"threads_started":1}
I20260810 11:45:02.565749 29863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:02.566126 29863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:02.566236 29863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a58cd892e2246c9a8d86338b9725210, State: Initialized, Role: FOLLOWER
I20260810 11:45:02.566911 29863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:02.567163 29863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:45:02.567395 29863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:45:02.567598 29863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:02.569864 29863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:02.570389 29863 leader_election.cc:302] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a58cd892e2246c9a8d86338b9725210; no voters:
I20260810 11:45:02.571051 29863 leader_election.cc:288] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:45:02.571208 29870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:45:02.571928 29870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 LEADER]: Becoming Leader. State: Replica: 5a58cd892e2246c9a8d86338b9725210, State: Running, Role: LEADER
I20260810 11:45:02.572613 29870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:02.573352 29863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:45:02.575775 29872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a58cd892e2246c9a8d86338b9725210. Latest consensus state: current_term: 1 leader_uuid: "5a58cd892e2246c9a8d86338b9725210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } } }
I20260810 11:45:02.576231 29872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:02.576917 29871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a58cd892e2246c9a8d86338b9725210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } } }
I20260810 11:45:02.577174 29871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:02.577147 29880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:45:02.583052 29880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:45:02.592023 29880 catalog_manager.cc:1347] Generated new cluster ID: d4cf3f764a95495c91ebf4113ace2039
I20260810 11:45:02.592147 29880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:45:02.619639 29880 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:45:02.621697 29880 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:45:02.634639 29880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Generated new TSK 0
I20260810 11:45:02.635813 29880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:45:02.717859 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:03.075181 29893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:03.075630 29893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:03.075819 29893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:03.085265 29893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:03.085407 29893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:03.085664 29893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:45:03.095700 29893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:03.097100 29893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:03.098498 29893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:03.111953 29898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:03.111884 29899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:03.112454 29901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:03.112747 29893 server_base.cc:1034] running on GCE node
I20260810 11:45:03.113343 29893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:03.114403 29893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:03.115628 29893 hybrid_clock.cc:648] HybridClock initialized: now 1786362303115570 us; error 44 us; skew 500 ppm
I20260810 11:45:03.115976 29893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:03 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:03.119447 29893 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:45:03.120946 29893 webserver.cc:460] Webserver started at http://127.20.93.129:45361/ using document root <none> and password file <none>
I20260810 11:45:03.121686 29893 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:03.121793 29893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:03.122058 29893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:03.124966 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "c8693d25aeea4d688e45bba758eb823b"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "e79bf9dddb8feb494967b762281a4e36"
server_key_iv: "fa89391c51fc73286c38ce02d5115bbd"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.125815 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "c8693d25aeea4d688e45bba758eb823b"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "e79bf9dddb8feb494967b762281a4e36"
server_key_iv: "fa89391c51fc73286c38ce02d5115bbd"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.131345 29893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 11:45:03.135010 29910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.136502 29893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:45:03.136703 29893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "c8693d25aeea4d688e45bba758eb823b"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "e79bf9dddb8feb494967b762281a4e36"
server_key_iv: "fa89391c51fc73286c38ce02d5115bbd"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.136943 29893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:03.178869 29893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:03.183576 29893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:03.183907 29893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:03.184968 29893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:03.186482 29893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:03.186589 29893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.186743 29893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:03.186815 29893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.221908 29893 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:37435
I20260810 11:45:03.221962 30030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:37435 every 8 connection(s)
I20260810 11:45:03.223284 29893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:45:03.228082 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 29893
I20260810 11:45:03.228322 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:45:03.229193 20854 external_mini_cluster.cc:1428] Setting key cdb1d3f7f1a5c163634d9d480230641c
Aug 10 11:45:03 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:03.248458 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:03.226106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:56063 (local address 127.20.93.190:46325)
0810 11:45:03.226580 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:03.226586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:03.227715 (+ 1129us) server_negotiation.cc:407] Connection header received
0810 11:45:03.230532 (+ 2817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:03.230538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:03.230645 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:03.230771 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:03.232941 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.233775 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.236355 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.236769 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.236874 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:03.241689 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:03.241695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:03.241699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:03.241737 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:03.244605 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:03.245484 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:03.245494 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:03.245496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:03.245711 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:03.246177 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:03.246182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:03.246185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:03.246503 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:03.246667 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:03.247619 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 11:45:03.247968 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":237,"threads_started":1}
I20260810 11:45:03.249130 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:03.226733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:56063)
0810 11:45:03.227460 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:03.227548 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:03.230122 (+ 2574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:03.231001 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:03.231012 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:03.231819 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:03.232738 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:03.232769 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.233954 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.233963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:03.236175 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:03.236195 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.237018 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.237028 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:03.237391 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:03.238249 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:03.238286 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:03.241382 (+ 3096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:03.244811 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:03.244820 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:03.244847 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:03.245331 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:03.245827 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:03.245833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:03.245837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:03.246051 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:03.246658 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:03.246666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:03.247094 (+ 428us) client_negotiation.cc:769] Sending connection context
0810 11:45:03.247459 (+ 365us) client_negotiation.cc:240] Negotiation successful
0810 11:45:03.247902 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":287,"threads_started":1}
I20260810 11:45:03.250952 30031 heartbeater.cc:347] Connected to a master server at 127.20.93.190:46325
I20260810 11:45:03.251307 30031 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:03.252085 30031 heartbeater.cc:511] Master 127.20.93.190:46325 requested a full tablet report, sending...
I20260810 11:45:03.254464 29825 ts_manager.cc:194] Registered new tserver with Master: c8693d25aeea4d688e45bba758eb823b (127.20.93.129:37435)
I20260810 11:45:03.255963 29825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:56063
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:45:03.295079 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:03.643349 30037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:03.643682 30037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:03.643836 30037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:03.651512 30037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:03.651633 30037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:03.651748 30037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:45:03.661929 30037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:03.663328 30037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:03.664963 30037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:03.679107 30042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:03.679170 30045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:03.679193 30043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:03.680236 30037 server_base.cc:1034] running on GCE node
I20260810 11:45:03.680838 30037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:03.681842 30037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:03.683096 30037 hybrid_clock.cc:648] HybridClock initialized: now 1786362303683025 us; error 33 us; skew 500 ppm
I20260810 11:45:03.683529 30037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:03 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:03.687075 30037 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:45:03.688619 30037 webserver.cc:460] Webserver started at http://127.20.93.130:37905/ using document root <none> and password file <none>
I20260810 11:45:03.689313 30037 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:03.689415 30037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:03.689780 30037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:03.692759 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "6ffbe70f8a8942e993c60b9e6358ddc4"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "10681030403cbd25426b32fbca2738a8"
server_key_iv: "8d998b42cfff15b8a5e8b0cbc8cd56a6"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.693542 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "6ffbe70f8a8942e993c60b9e6358ddc4"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "10681030403cbd25426b32fbca2738a8"
server_key_iv: "8d998b42cfff15b8a5e8b0cbc8cd56a6"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.698913 30037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:45:03.702425 30053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.703811 30037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:45:03.703953 30037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "6ffbe70f8a8942e993c60b9e6358ddc4"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "10681030403cbd25426b32fbca2738a8"
server_key_iv: "8d998b42cfff15b8a5e8b0cbc8cd56a6"
server_key_version: "encryptionkey@0"
I20260810 11:45:03.704124 30037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:03.721112 30037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:03.726089 30037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:03.726341 30037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:03.727299 30037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:03.728711 30037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:03.728798 30037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.728889 30037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:03.728933 30037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:03.763197 30037 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44543
I20260810 11:45:03.763290 30176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44543 every 8 connection(s)
I20260810 11:45:03.764603 30037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:45:03.774437 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30037
I20260810 11:45:03.774667 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:45:03.775290 20854 external_mini_cluster.cc:1428] Setting key 3a423a1a6a16970f684118d1e00d1282
Aug 10 11:45:03 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:03.788198 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:03.767332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:48893 (local address 127.20.93.190:46325)
0810 11:45:03.767803 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:03.767807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:03.768992 (+ 1185us) server_negotiation.cc:407] Connection header received
0810 11:45:03.770883 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:03.770887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:03.770957 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:03.771077 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:03.772590 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.773355 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.775780 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.776439 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.776588 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:03.780855 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:03.780860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:03.780863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:03.780897 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:03.784592 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:03.785404 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:03.785412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:03.785418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:03.785631 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:03.786091 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:03.786099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:03.786105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:03.786515 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:03.786687 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:03.787423 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 11:45:03.787715 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":248,"threads_started":1}
I20260810 11:45:03.788918 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:03.768020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:48893)
0810 11:45:03.768816 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:03.768863 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:03.770587 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:03.771273 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:03.771281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:03.771827 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:03.772395 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:03.772412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.773514 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.773518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:03.775626 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:03.775642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:03.776715 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:03.776729 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:03.776950 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:03.777809 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:03.777848 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:03.780607 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:03.784772 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:03.784779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:03.784793 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:03.785261 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:03.785754 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:03.785758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:03.785760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:03.785964 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:03.786658 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:03.786665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:03.787031 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 11:45:03.787283 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 11:45:03.787707 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":315,"threads_started":1}
I20260810 11:45:03.790525 30177 heartbeater.cc:347] Connected to a master server at 127.20.93.190:46325
I20260810 11:45:03.790881 30177 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:03.791659 30177 heartbeater.cc:511] Master 127.20.93.190:46325 requested a full tablet report, sending...
I20260810 11:45:03.793315 29825 ts_manager.cc:194] Registered new tserver with Master: 6ffbe70f8a8942e993c60b9e6358ddc4 (127.20.93.130:44543)
I20260810 11:45:03.794656 29825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:48893
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:45:03.845669 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:04.195257 30183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:04.195600 30183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:04.195798 30183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:04.203352 30183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:04.203519 30183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:04.203696 30183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:45:04.214139 30183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:04.215539 30183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:04.216960 30183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:04.230829 30189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:04.230883 30191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:04.230921 30188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:04.231814 30183 server_base.cc:1034] running on GCE node
I20260810 11:45:04.232380 30183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:04.233403 30183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:04.234632 30183 hybrid_clock.cc:648] HybridClock initialized: now 1786362304234570 us; error 55 us; skew 500 ppm
I20260810 11:45:04.234928 30183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:04 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:04.238224 30183 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:45:04.239778 30183 webserver.cc:460] Webserver started at http://127.20.93.131:45575/ using document root <none> and password file <none>
I20260810 11:45:04.240561 30183 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:04.240705 30183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:04.241055 30183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:04.244144 30183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "250e97fcb0b0495193f49ba6b305291d"
format_stamp: "Formatted at 2026-08-10 11:45:04 on dist-test-slave-njxd"
server_key: "52571fb87722fd0edcb6f5845a6c3d52"
server_key_iv: "3b24933b9aeca095d3b02dd0e720dfc9"
server_key_version: "encryptionkey@0"
I20260810 11:45:04.244947 30183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "250e97fcb0b0495193f49ba6b305291d"
format_stamp: "Formatted at 2026-08-10 11:45:04 on dist-test-slave-njxd"
server_key: "52571fb87722fd0edcb6f5845a6c3d52"
server_key_iv: "3b24933b9aeca095d3b02dd0e720dfc9"
server_key_version: "encryptionkey@0"
I20260810 11:45:04.250384 30183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260810 11:45:04.253989 30201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:04.255398 30183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 11:45:04.255579 30183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "250e97fcb0b0495193f49ba6b305291d"
format_stamp: "Formatted at 2026-08-10 11:45:04 on dist-test-slave-njxd"
server_key: "52571fb87722fd0edcb6f5845a6c3d52"
server_key_iv: "3b24933b9aeca095d3b02dd0e720dfc9"
server_key_version: "encryptionkey@0"
I20260810 11:45:04.255810 30183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:04.259217 30031 heartbeater.cc:503] Master 127.20.93.190:46325 was elected leader, sending a full tablet report...
I20260810 11:45:04.284433 30183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:04.289127 30183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:04.289466 30183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:04.290566 30183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:04.292088 30183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:04.292200 30183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:04.292351 30183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:04.292428 30183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:04.327121 30183 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:35069
I20260810 11:45:04.327176 30324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:35069 every 8 connection(s)
I20260810 11:45:04.328476 30183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:45:04.335450 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30183
I20260810 11:45:04.335604 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:45:04.336063 20854 external_mini_cluster.cc:1428] Setting key 787d35925d08d724f69cdfae70461778
Aug 10 11:45:04 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:04.350044 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:04.331204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:53515 (local address 127.20.93.190:46325)
0810 11:45:04.331708 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:04.331716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:04.332724 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 11:45:04.334625 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:04.334629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:04.334700 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:04.334799 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:04.336617 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.337671 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.339613 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.340030 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.340170 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:04.343902 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:04.343907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:04.343910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:04.343944 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:04.346751 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.347548 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.347554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.347557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.347700 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.348155 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.348163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.348167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.348452 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:04.348641 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:04.349363 (+ 722us) server_negotiation.cc:299] Negotiation successful
0810 11:45:04.349648 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":225,"threads_started":1}
I20260810 11:45:04.351287 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:04.331795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:53515)
0810 11:45:04.332549 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:04.332595 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:04.334332 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:04.335053 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:04.335076 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:04.335722 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:04.336404 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:04.336423 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.337835 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.337839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:04.339442 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:04.339460 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.340277 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.340287 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:04.340558 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:04.341317 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:04.341341 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:04.343673 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:04.346912 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:04.346921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:04.346939 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:04.347419 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:04.347820 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:04.347825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:04.347829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:04.348029 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:04.348587 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:04.348596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:04.348965 (+ 369us) client_negotiation.cc:769] Sending connection context
0810 11:45:04.349264 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 11:45:04.349851 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":307,"threads_started":1}
I20260810 11:45:04.352950 30325 heartbeater.cc:347] Connected to a master server at 127.20.93.190:46325
I20260810 11:45:04.353292 30325 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:04.354161 30325 heartbeater.cc:511] Master 127.20.93.190:46325 requested a full tablet report, sending...
I20260810 11:45:04.355614 29825 ts_manager.cc:194] Registered new tserver with Master: 250e97fcb0b0495193f49ba6b305291d (127.20.93.131:35069)
I20260810 11:45:04.356444 29825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:53515
I20260810 11:45:04.364306 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
Entry for principal oryx/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab.
WARNING: no policy specified for oryx/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
Entry for principal oryx/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab.
WARNING: no policy specified for oryx/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
Entry for principal oryx/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab.
I20260810 11:45:04.545042 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29893
Aug 10 11:45:04 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:45:04.563030 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:04.546410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.20.93.129:37435)
0810 11:45:04.547371 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:04.547379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:04.547407 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:45:04.547472 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:04.547479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:04.547718 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:04.547857 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:04.549255 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.550420 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.551675 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.552265 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.552367 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:04.556743 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:04.556758 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:04.556774 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:04.556822 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:04.560029 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.560890 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.560898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.560900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.561043 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.561395 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.561398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.561400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.561853 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:04.562110 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:04.562386 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 11:45:04.562621 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":620,"threads_started":1}
I20260810 11:45:04.564313 30026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:04.648677 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29893
I20260810 11:45:04.669047 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30037
Aug 10 11:45:04 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:45:04.687129 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:04.670081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.20.93.130:44543)
0810 11:45:04.670664 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:04.670672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:04.670698 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:45:04.670778 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:04.670785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:04.671005 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:04.671147 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:04.672661 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.673753 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.675320 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.675871 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.675972 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:04.680725 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:04.680739 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:04.680755 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:04.680795 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:04.683863 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.684654 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.684662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.684664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.684810 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.685195 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.685198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.685200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.685547 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:04.686166 (+ 619us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:04.686499 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 11:45:04.686710 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":320,"threads_started":1}
I20260810 11:45:04.688501 30170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:04.773869 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30037
I20260810 11:45:04.792759 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30183
Aug 10 11:45:04 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:45:04.810572 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:04.793866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.20.93.131:35069)
0810 11:45:04.794372 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:04.794380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:04.794410 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:45:04.794471 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:04.794478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:04.794716 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:04.794900 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:04.796318 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.797336 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.798716 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:04.799305 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:04.799461 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:04.804337 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:04.804352 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:04.804368 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:04.804409 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:04.807425 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.808202 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.808211 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.808213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.808351 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:04.808716 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:04.808720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:04.808722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:04.809094 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:04.809349 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:04.809877 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 11:45:04.810171 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":232,"threads_started":1}
I20260810 11:45:04.811975 30320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:04.896935 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30183
I20260810 11:45:04.915592 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 29792
I20260810 11:45:04.916927 29858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:05.028108 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 29792
I20260810 11:45:05.052582 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:46325
--webserver_interface=127.20.93.190
--webserver_port=42551
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:46325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:05.425354 30345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:05.425766 30345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:05.425931 30345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:05.433454 30345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:05.433671 30345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:05.433772 30345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:05.433873 30345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:05.433941 30345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:05.444633 30345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:46325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:46325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=42551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:05.446338 30345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:05.447927 30345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:05.461022 30353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:05.461071 30351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:05.461203 30350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:05.462060 30345 server_base.cc:1034] running on GCE node
I20260810 11:45:05.462996 30345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:05.464604 30345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:05.465863 30345 hybrid_clock.cc:648] HybridClock initialized: now 1786362305465811 us; error 69 us; skew 500 ppm
I20260810 11:45:05.466265 30345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:05 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362305, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:05.473170 30345 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:45:05.474969 30345 webserver.cc:460] Webserver started at http://127.20.93.190:42551/ using document root <none> and password file <none>
I20260810 11:45:05.475767 30345 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:05.475941 30345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:05.482257 30345 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:45:05.485519 30360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:05.487314 30345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:45:05.487533 30345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "5a58cd892e2246c9a8d86338b9725210"
format_stamp: "Formatted at 2026-08-10 11:45:02 on dist-test-slave-njxd"
server_key: "ec26490a26a49204461943f1e5bf549b"
server_key_iv: "78c82ebf422a77f469af1dbd3a0f0d4d"
server_key_version: "encryptionkey@0"
I20260810 11:45:05.488632 30345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:05.504206 30345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:05.509289 30345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:05.509673 30345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:05.530624 30345 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:46325
I20260810 11:45:05.530645 30415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:46325 every 8 connection(s)
I20260810 11:45:05.532464 30345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:45:05.538096 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30345
I20260810 11:45:05.539233 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:37435
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:46325
--webserver_port=45361
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:45:05.541100 30417 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:45:05.546761 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Bootstrap starting.
I20260810 11:45:05.568833 30417 log.cc:824] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:05.579116 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:45:05.579744 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Bootstrap complete.
I20260810 11:45:05.589782 30417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:05.590718 30417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a58cd892e2246c9a8d86338b9725210, State: Initialized, Role: FOLLOWER
I20260810 11:45:05.591482 30417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:05.591747 30417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:45:05.591871 30417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:45:05.592096 30417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:45:05.596624 30417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:05.597247 30417 leader_election.cc:302] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a58cd892e2246c9a8d86338b9725210; no voters:
I20260810 11:45:05.598014 30417 leader_election.cc:288] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:45:05.598165 30421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:45:05.601056 30417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:45:05.605634 30421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [term 2 LEADER]: Becoming Leader. State: Replica: 5a58cd892e2246c9a8d86338b9725210, State: Running, Role: LEADER
I20260810 11:45:05.606528 30421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } }
I20260810 11:45:05.609697 30422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5a58cd892e2246c9a8d86338b9725210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } } }
I20260810 11:45:05.610134 30422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:05.609656 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a58cd892e2246c9a8d86338b9725210. Latest consensus state: current_term: 2 leader_uuid: "5a58cd892e2246c9a8d86338b9725210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a58cd892e2246c9a8d86338b9725210" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 46325 } } }
I20260810 11:45:05.610782 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:05.621955 30434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:45:05.627919 30434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:45:05.629400 30434 catalog_manager.cc:1259] Loaded cluster ID: d4cf3f764a95495c91ebf4113ace2039
I20260810 11:45:05.629532 30434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:45:05.633723 30434 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:45:05.635269 30434 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5a58cd892e2246c9a8d86338b9725210: Loaded TSK: 0
I20260810 11:45:05.636900 30434 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:45:05.913834 30419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:05.914212 30419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:05.914389 30419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:05.922452 30419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:05.922646 30419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:05.922843 30419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:45:05.933555 30419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.129.keytab
--principal=oryx/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:37435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=45361
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:05.935134 30419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:05.936494 30419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:05.952701 30447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:05.952944 30449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:05.953630 30445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:05.959349 30419 server_base.cc:1034] running on GCE node
I20260810 11:45:05.960245 30419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:05.961464 30419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:05.962782 30419 hybrid_clock.cc:648] HybridClock initialized: now 1786362305962720 us; error 61 us; skew 500 ppm
I20260810 11:45:05.963147 30419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:05 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362305, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:05.966958 30419 init.cc:377] Logged in from keytab as oryx/127.20.93.129@KRBTEST.COM (short username oryx)
I20260810 11:45:05.969024 30419 webserver.cc:460] Webserver started at http://127.20.93.129:45361/ using document root <none> and password file <none>
I20260810 11:45:05.970046 30419 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:05.970239 30419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:05.976374 30419 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:45:05.979336 30456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:05.980859 30419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:45:05.981086 30419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "c8693d25aeea4d688e45bba758eb823b"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "e79bf9dddb8feb494967b762281a4e36"
server_key_iv: "fa89391c51fc73286c38ce02d5115bbd"
server_key_version: "encryptionkey@0"
I20260810 11:45:05.982167 30419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:06.010581 30419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:06.015509 30419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:06.015838 30419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:06.017242 30419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:06.018746 30419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:06.018841 30419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:06.018963 30419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:06.019047 30419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:06.053753 30419 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:37435
I20260810 11:45:06.053841 30580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:37435 every 8 connection(s)
I20260810 11:45:06.055272 30419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:45:06.062811 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30419
I20260810 11:45:06.063499 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:44543
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:46325
--webserver_port=37905
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:45:06 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:06 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:45:06.427357 30585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:06.427724 30585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:06.427892 30585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:06.435566 30585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:06.435735 30585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:06.435912 30585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:45:06.446157 30585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.130.keytab
--principal=oryx/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:44543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=37905
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:06.447590 30585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:06.448999 30585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:06.468106 30594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:06.468124 30596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:06.468258 30592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:06.469192 30585 server_base.cc:1034] running on GCE node
I20260810 11:45:06.469964 30585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:06.471088 30585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:06.472555 30585 hybrid_clock.cc:648] HybridClock initialized: now 1786362306472463 us; error 87 us; skew 500 ppm
I20260810 11:45:06.472914 30585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:06 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362306, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:06.476807 30585 init.cc:377] Logged in from keytab as oryx/127.20.93.130@KRBTEST.COM (short username oryx)
I20260810 11:45:06.478682 30585 webserver.cc:460] Webserver started at http://127.20.93.130:37905/ using document root <none> and password file <none>
I20260810 11:45:06.479578 30585 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:06.479737 30585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:06.486614 30585 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260810 11:45:06.490067 30604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:06.491909 30585 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:45:06.492172 30585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "6ffbe70f8a8942e993c60b9e6358ddc4"
format_stamp: "Formatted at 2026-08-10 11:45:03 on dist-test-slave-njxd"
server_key: "10681030403cbd25426b32fbca2738a8"
server_key_iv: "8d998b42cfff15b8a5e8b0cbc8cd56a6"
server_key_version: "encryptionkey@0"
I20260810 11:45:06.493207 30585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:06.514575 30585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:06.519326 30585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:06.519635 30585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:06.520797 30585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:06.522483 30585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:06.522611 30585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:06.522761 30585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:06.522851 30585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:06.573359 30585 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44543
I20260810 11:45:06.573401 30728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44543 every 8 connection(s)
I20260810 11:45:06.574975 30585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:45:06.578823 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30585
I20260810 11:45:06.579805 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:35069
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:46325
--webserver_port=45575
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:45:06 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:06 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:45:06.951378 30733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:06.951740 30733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:06.951924 30733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:06.959753 30733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:06.959918 30733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:06.960099 30733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:45:06.970335 30733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/krb5kdc/oryx_127.20.93.131.keytab
--principal=oryx/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:35069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=45575
--tserver_master_addrs=127.20.93.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:06.971769 30733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:06.973107 30733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:06.990694 30741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:06.995668 30740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:06.999637 30744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:07.000087 30733 server_base.cc:1034] running on GCE node
I20260810 11:45:07.000802 30733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:07.002055 30733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:07.003275 30733 hybrid_clock.cc:648] HybridClock initialized: now 1786362307003205 us; error 60 us; skew 500 ppm
I20260810 11:45:07.003614 30733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362307, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:07.007484 30733 init.cc:377] Logged in from keytab as oryx/127.20.93.131@KRBTEST.COM (short username oryx)
I20260810 11:45:07.017962 30733 webserver.cc:460] Webserver started at http://127.20.93.131:45575/ using document root <none> and password file <none>
I20260810 11:45:07.018780 30733 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:07.018944 30733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:07.024622 30733 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 11:45:07.027631 30750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:07.029163 30733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:45:07.029340 30733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "250e97fcb0b0495193f49ba6b305291d"
format_stamp: "Formatted at 2026-08-10 11:45:04 on dist-test-slave-njxd"
server_key: "52571fb87722fd0edcb6f5845a6c3d52"
server_key_iv: "3b24933b9aeca095d3b02dd0e720dfc9"
server_key_version: "encryptionkey@0"
I20260810 11:45:07.030383 30733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:07.078392 30733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:07.083156 30733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:07.083546 30733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:07.084685 30733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:07.086355 30733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:07.086478 30733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:07.086621 30733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:07.086712 30733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:07.138448 30733 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:35069
I20260810 11:45:07.138485 30872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:35069 every 8 connection(s)
I20260810 11:45:07.140060 30733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:45:07.145601 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30733
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:07.187161 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:07.147871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.20.93.190:46325)
0810 11:45:07.158251 (+ 10380us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:07.158262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:07.158297 (+ 35us) server_negotiation.cc:407] Connection header received
0810 11:45:07.158392 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:07.158401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:07.158523 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:07.158715 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:07.160823 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.162792 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.165835 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.166817 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.166957 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.173050 (+ 6093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:07.173066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:07.173117 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:07.173187 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:07.178842 (+ 5655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:07.180265 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:07.180280 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:07.180296 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:07.180619 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:07.181280 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:07.181288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:07.181294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:07.182281 (+ 987us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:07.182688 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:07.184737 (+ 2049us) server_negotiation.cc:299] Negotiation successful
0810 11:45:07.185490 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10153,"thread_start_us":521,"threads_started":1}
W20260810 11:45:07.477442 30584 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.408s user 0.000s sys 0.017s
W20260810 11:45:07.479080 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:06.058520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41541 (local address 127.20.93.190:46325)
0810 11:45:06.059389 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:06.059405 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:06.059619 (+ 214us) server_negotiation.cc:407] Connection header received
0810 11:45:06.061851 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:06.061864 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:06.062195 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:06.062548 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:06.064218 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.065509 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.067109 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.067782 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.067993 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.478726 (+1410733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:41541: BlockingRecv error: recv got EOF from 127.20.93.129:41541 (error 108)
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":336,"threads_started":1}
I20260810 11:45:07.479640 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:06.058587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:41541)
0810 11:45:06.059434 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:06.059480 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:06.061257 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:06.062704 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:06.062718 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:06.063357 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:06.064007 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:06.064026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.065718 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.065723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:06.066960 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:06.066972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.068023 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.068030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:06.068313 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:06.069015 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:06.069039 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:07.478419 (+1409380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":337,"threads_started":1}
W20260810 11:45:07.480604 30581 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:46325 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.93.190:46325: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:07.492184 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:07.481350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:55969)
0810 11:45:07.481640 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:07.481660 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:07.481820 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:07.482301 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:07.482306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:07.482749 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:07.483191 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.483206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.484493 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.484498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.486321 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.486334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.487183 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.487189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.487363 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.488026 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:07.488044 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:07.491784 (+ 3740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260810 11:45:07.492350 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:07.481412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:55969 (local address 127.20.93.190:46325)
0810 11:45:07.481626 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:07.481633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:07.481745 (+ 112us) server_negotiation.cc:407] Connection header received
0810 11:45:07.481951 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:07.481957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:07.482045 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:07.482142 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:07.483355 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.484130 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.486461 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.486946 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.487161 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.492051 (+ 4890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:55969: BlockingRecv error: recv got EOF from 127.20.93.129:55969 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:07 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:07.503659 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:07.493210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:46947)
0810 11:45:07.493615 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:07.493634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:07.493772 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:07.494156 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:07.494162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:07.494500 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:07.494668 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.494677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.495704 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.495710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.497769 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.497785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.498673 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.498677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.498811 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.499446 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:07.499463 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:07.503354 (+ 3891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263}
W20260810 11:45:07.503998 30581 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:45:07.504040 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:07.493290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46947 (local address 127.20.93.190:46325)
0810 11:45:07.493456 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:07.493460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:07.493712 (+ 252us) server_negotiation.cc:407] Connection header received
0810 11:45:07.493898 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:07.493903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:07.493989 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:07.494094 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:07.494812 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.495550 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.497931 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.498402 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.498475 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.503564 (+ 5089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:46947: BlockingRecv error: recv got EOF from 127.20.93.129:46947 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 11:45:08.094390 30732 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.488s user 0.005s sys 0.015s
W20260810 11:45:08.096283 30731 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:06.578153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:41315 (local address 127.20.93.190:46325)
0810 11:45:06.595642 (+ 17489us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:06.595653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:06.595685 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:45:06.595780 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:06.595788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:06.595930 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:06.596103 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:06.598687 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.600534 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.603423 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.604460 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.604670 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.095963 (+1491293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:41315: BlockingRecv error: recv got EOF from 127.20.93.130:41315 (error 108)
Metrics: {"server-negotiator.queue_time_us":17257,"thread_start_us":17069,"threads_started":1}
I20260810 11:45:08.097720 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:06.579236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:41315)
0810 11:45:06.588804 (+ 9568us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:06.588886 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:06.594446 (+ 5560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:06.596487 (+ 2041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:06.596505 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:06.597427 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:06.598368 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:06.598398 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.600790 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.600798 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:06.603201 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:06.603221 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:06.604773 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:06.604783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:06.605183 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:06.606311 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:06.606354 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.095822 (+1489468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":9023,"thread_start_us":12579,"threads_started":1}
W20260810 11:45:08.098804 30729 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:46325 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.93.190:46325: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:45:08.111773 30731 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:08.099772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:57123 (local address 127.20.93.190:46325)
0810 11:45:08.099949 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:08.099953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:08.099977 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:45:08.100244 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:08.100248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:08.100312 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:08.100427 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:08.101654 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.102744 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.104736 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.105407 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.105553 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.111570 (+ 6017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:57123: BlockingRecv error: recv got EOF from 127.20.93.130:57123 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:45:08.111811 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:08.099587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:57123)
0810 11:45:08.099874 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:08.099906 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:08.100083 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:08.100701 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:08.100709 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:08.101198 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:08.101413 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.101427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.103191 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.103199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.104550 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.104569 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.105650 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.105657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.105880 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.106571 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:08.106590 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.111312 (+ 4722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:08.127419 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:08.112772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:39019)
0810 11:45:08.113031 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:08.113053 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:08.113189 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:08.113996 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:08.114005 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:08.114743 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:08.115076 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.115096 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.117401 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.117411 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.119807 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.119830 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.120735 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.120746 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.121048 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.121953 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:08.121993 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.127110 (+ 5117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118}
W20260810 11:45:08.127782 30729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:45:08.127876 30731 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:08.112987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:39019 (local address 127.20.93.190:46325)
0810 11:45:08.113264 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:08.113274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:08.113305 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:45:08.113398 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:08.113406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:08.113525 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:08.113713 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:08.115364 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.117056 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.120078 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.120531 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.120646 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.127597 (+ 6951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:39019: BlockingRecv error: recv got EOF from 127.20.93.130:39019 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 11:45:08.457983 30875 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.287s user 0.011s sys 0.015s
W20260810 11:45:08.459297 30878 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:07.146605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:58285 (local address 127.20.93.190:46325)
0810 11:45:07.148604 (+ 1999us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:07.148611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:07.158680 (+ 10069us) server_negotiation.cc:407] Connection header received
0810 11:45:07.159963 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:07.159970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:07.160065 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:07.160177 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:07.162229 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.164193 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.167944 (+ 3751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.168915 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.169044 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.459036 (+1289992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:58285: BlockingRecv error: recv got EOF from 127.20.93.131:58285 (error 108)
Metrics: {"server-negotiator.queue_time_us":1882,"thread_start_us":11189,"threads_started":1}
I20260810 11:45:08.459784 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:07.144385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:58285)
0810 11:45:07.150533 (+ 6148us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:07.150634 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:07.159453 (+ 8819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:07.160701 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:07.160730 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:07.161340 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:07.162046 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.162064 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.164497 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.164506 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.167694 (+ 3188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:07.167716 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:07.169621 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:07.169633 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:07.170165 (+ 532us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:07.171230 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:07.171286 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.458779 (+1287493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5632,"thread_start_us":5287,"threads_started":1}
W20260810 11:45:08.460745 30873 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:46325 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.93.190:46325: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:08.471067 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:08.461536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:56093)
0810 11:45:08.461786 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:08.461805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:08.461908 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:08.462280 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:08.462285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:08.462864 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:08.463109 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.463123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.464389 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.464395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.465468 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.465482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.466262 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.466268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.466461 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.467065 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:08.467084 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.470833 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260810 11:45:08.471410 30878 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:08.461680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:56093 (local address 127.20.93.190:46325)
0810 11:45:08.461841 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:08.461846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:08.461864 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:45:08.462019 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:08.462023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:08.462087 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:08.462185 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:08.463277 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.464232 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.465643 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.466063 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.466221 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.471135 (+ 4914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:56093: BlockingRecv error: recv got EOF from 127.20.93.131:56093 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:08.481933 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:08.472059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:35089)
0810 11:45:08.472231 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:08.472256 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:08.472348 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:08.472776 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:08.472779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:08.473095 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:08.473272 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.473282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.474424 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.474428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.475721 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.475737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.476869 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.476875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.477113 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.477857 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:08.477877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.481611 (+ 3734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:45:08.482162 30878 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:08.472156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35089 (local address 127.20.93.190:46325)
0810 11:45:08.472432 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:08.472437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:08.472454 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:45:08.472505 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:08.472508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:08.472583 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:08.472700 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:08.473423 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.474248 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.475948 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.476657 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.476795 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.481875 (+ 5080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:35089: BlockingRecv error: recv got EOF from 127.20.93.131:35089 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260810 11:45:08.482407 30873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:08 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:08.514631 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:08.504563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:40425)
0810 11:45:08.505179 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:08.505195 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:08.505319 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:08.505829 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:08.505834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:08.506271 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:08.506425 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.506433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.507560 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.507565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.508564 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:08.508577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.509344 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.509349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:08.509489 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.510238 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:08.510258 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:08.514263 (+ 4005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":365,"threads_started":1}
W20260810 11:45:08.514804 30878 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:08.504691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:40425 (local address 127.20.93.190:46325)
0810 11:45:08.504863 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:08.504867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:08.505280 (+ 413us) server_negotiation.cc:407] Connection header received
0810 11:45:08.505466 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:08.505471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:08.505631 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:08.505750 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:08.506589 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.507390 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.508712 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:08.509161 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:08.509257 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:08.514496 (+ 5239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:40425: BlockingRecv error: recv got EOF from 127.20.93.129:40425 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:09.138360 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:09.128521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:33843)
0810 11:45:09.129078 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:09.129099 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:09.129238 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:09.129559 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:09.129562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:09.130069 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:09.130229 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.130237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.131257 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.131260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.132222 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.132233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.132968 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.132975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.133161 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.133878 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:09.133904 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:09.138107 (+ 4203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":329,"threads_started":1}
W20260810 11:45:09.138650 30882 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:09.128558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:33843 (local address 127.20.93.190:46325)
0810 11:45:09.129142 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:09.129147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:09.129164 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:45:09.129349 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:09.129352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:09.129422 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:09.129513 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:09.130359 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.131130 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.132343 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.132771 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.132911 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.138364 (+ 5453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:33843: BlockingRecv error: recv got EOF from 127.20.93.130:33843 (error 108)
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:09.493114 30884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:09.483036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:42589)
0810 11:45:09.483639 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:09.483656 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:09.483788 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:09.484115 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:09.484119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:09.484575 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:09.484731 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.484740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.485847 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.485855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.487068 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.487087 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.487810 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.487815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.487937 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.488557 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:09.488576 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:09.492809 (+ 4233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":389,"threads_started":1}
W20260810 11:45:09.493358 30882 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:09.483155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:42589 (local address 127.20.93.190:46325)
0810 11:45:09.483456 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:09.483463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:09.483754 (+ 291us) server_negotiation.cc:407] Connection header received
0810 11:45:09.483902 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:09.483906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:09.483974 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:09.484064 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:09.484870 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.485663 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.487230 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.487643 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.487743 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.493051 (+ 5308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:42589: BlockingRecv error: recv got EOF from 127.20.93.131:42589 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:09 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:09.525640 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:09.515711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:47025)
0810 11:45:09.516280 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:09.516301 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:09.516445 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:09.516836 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:09.516841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:09.517335 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:09.517528 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.517537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.518676 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.518682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.519678 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:09.519702 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.520470 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.520476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:09.520670 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.521400 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:09.521429 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:09.525241 (+ 3812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":324,"threads_started":1}
W20260810 11:45:09.525835 30882 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:09.515817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:47025 (local address 127.20.93.190:46325)
0810 11:45:09.516025 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:09.516033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:09.516373 (+ 340us) server_negotiation.cc:407] Connection header received
0810 11:45:09.516587 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:09.516592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:09.516681 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:09.516795 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:09.517713 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.518518 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.519844 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:09.520301 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:09.520447 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:09.525493 (+ 5046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:47025: BlockingRecv error: recv got EOF from 127.20.93.129:47025 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:10.148800 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:10.139399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:49693)
0810 11:45:10.140015 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:10.140039 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:10.140172 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:10.140484 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:10.140488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:10.140908 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:10.141079 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.141087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.142089 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.142093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.143013 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.143025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.143875 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.143881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.144071 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.144747 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:10.144775 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:10.148538 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":301,"threads_started":1}
W20260810 11:45:10.149012 30887 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:10.139603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:49693 (local address 127.20.93.190:46325)
0810 11:45:10.140146 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:10.140151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:10.140171 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:45:10.140275 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:10.140279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:10.140346 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:10.140439 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:10.141216 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.141956 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.143181 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.143708 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.143811 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.148763 (+ 4952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:49693: BlockingRecv error: recv got EOF from 127.20.93.130:49693 (error 108)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":312,"threads_started":1}
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:10.511912 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:10.494454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:60303)
0810 11:45:10.495003 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:10.495022 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:10.495162 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:10.495492 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:10.495495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:10.495931 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:10.496117 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.496127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.497430 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.497436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.498913 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.498953 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.499886 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.499892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.500114 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.506945 (+ 6831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:10.506970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:10.511569 (+ 4599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":315,"threads_started":1}
W20260810 11:45:10.512108 30887 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:10.494669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:60303 (local address 127.20.93.190:46325)
0810 11:45:10.494926 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:10.494933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:10.495116 (+ 183us) server_negotiation.cc:407] Connection header received
0810 11:45:10.495278 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:10.495282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:10.495349 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:10.495437 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:10.496273 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.497216 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.499080 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.499666 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.499775 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.511802 (+ 12027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:60303: BlockingRecv error: recv got EOF from 127.20.93.131:60303 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:10 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:10.536378 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:10.526714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:50853)
0810 11:45:10.527256 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:10.527271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:10.527431 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:10.527808 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:10.527812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:10.528221 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:10.528400 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.528409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.529451 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.529457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.530444 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:10.530455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.531234 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.531239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:10.531417 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.532059 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:10.532076 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:10.536084 (+ 4008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":281,"threads_started":1}
W20260810 11:45:10.536629 30887 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:10.526786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:50853 (local address 127.20.93.190:46325)
0810 11:45:10.527020 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:10.527029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:10.527357 (+ 328us) server_negotiation.cc:407] Connection header received
0810 11:45:10.527557 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:10.527569 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:10.527651 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:10.527776 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:10.528560 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.529314 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.530588 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:10.531065 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:10.531208 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:10.536328 (+ 5120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:50853: BlockingRecv error: recv got EOF from 127.20.93.129:50853 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:11.159822 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:11.150028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:57451)
0810 11:45:11.150601 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:11.150618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:11.150740 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:11.151193 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:11.151198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:11.151710 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:11.151931 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.151941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.153259 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.153262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.154311 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.154323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.155007 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.155010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.155131 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.155731 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:11.155748 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:11.159452 (+ 3704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":326,"threads_started":1}
W20260810 11:45:11.160043 30891 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:11.150131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:57451 (local address 127.20.93.190:46325)
0810 11:45:11.150653 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:11.150659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:11.150682 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:45:11.150876 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:11.150881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:11.150984 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:11.151115 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:11.152091 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.153113 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.154441 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.154881 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.154978 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.159761 (+ 4783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:57451: BlockingRecv error: recv got EOF from 127.20.93.130:57451 (error 108)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":301,"threads_started":1}
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:11.522596 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:11.513066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:41577)
0810 11:45:11.513616 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:11.513632 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:11.513806 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:11.514160 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:11.514166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:11.514579 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:11.514766 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.514776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.515842 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.515848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.516777 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.516789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.517489 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.517495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.517719 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.518318 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:11.518337 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:11.522263 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
W20260810 11:45:11.522781 30891 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:11.513176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:41577 (local address 127.20.93.190:46325)
0810 11:45:11.513408 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:11.513417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:11.513719 (+ 302us) server_negotiation.cc:407] Connection header received
0810 11:45:11.513937 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:11.513942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:11.514018 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:11.514121 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:11.514921 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.515694 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.516923 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.517340 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.517474 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.522532 (+ 5058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:41577: BlockingRecv error: recv got EOF from 127.20.93.131:41577 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:11 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:11.547386 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:11.537719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:52845)
0810 11:45:11.538241 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:11.538257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:11.538408 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:11.538782 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:11.538786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:11.539212 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:11.539381 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.539390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.540552 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.540562 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.541640 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:11.541660 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.542365 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.542371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:11.542556 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.543201 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:11.543222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:11.547005 (+ 3783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":271,"threads_started":1}
W20260810 11:45:11.547497 30891 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:11.537793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:52845 (local address 127.20.93.190:46325)
0810 11:45:11.538073 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:11.538080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:11.538338 (+ 258us) server_negotiation.cc:407] Connection header received
0810 11:45:11.538528 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:11.538534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:11.538626 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:11.538732 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:11.539523 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.540406 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.541784 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:11.542208 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:11.542351 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:11.547208 (+ 4857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:52845: BlockingRecv error: recv got EOF from 127.20.93.129:52845 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:12.170639 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:12.161005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:54525)
0810 11:45:12.161475 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:12.161493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:12.161692 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:12.162137 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:12.162141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:12.162523 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:12.162807 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.162825 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.164095 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.164100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.165282 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.165297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.165970 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.165974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.166097 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.166770 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:12.166789 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:12.170344 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":252,"threads_started":1}
W20260810 11:45:12.170948 30895 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:12.161106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:54525 (local address 127.20.93.190:46325)
0810 11:45:12.161840 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:12.161845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:12.161863 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:45:12.161913 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:12.161917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:12.161991 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:12.162083 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:12.162978 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.163913 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.165413 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.165847 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.165940 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.170560 (+ 4620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:54525: BlockingRecv error: recv got EOF from 127.20.93.130:54525 (error 108)
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":527,"threads_started":1}
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:12.533538 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:12.524109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:35791)
0810 11:45:12.524706 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:12.524724 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:12.524896 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:12.525263 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:12.525268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:12.525735 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:12.525906 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.525916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.526995 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.527001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.528062 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.528084 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.528870 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.528874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.529015 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.529694 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:12.529723 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:12.533288 (+ 3565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":312,"threads_started":1}
W20260810 11:45:12.533870 30895 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:12.524189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35791 (local address 127.20.93.190:46325)
0810 11:45:12.524402 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:12.524412 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:12.524828 (+ 416us) server_negotiation.cc:407] Connection header received
0810 11:45:12.525030 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:12.525034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:12.525098 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:12.525234 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:12.526062 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.526847 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.528277 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.528709 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.528830 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.533606 (+ 4776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:35791: BlockingRecv error: recv got EOF from 127.20.93.131:35791 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:12 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:12.558920 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:12.548684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:33795)
0810 11:45:12.549344 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:12.549368 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:12.549552 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:12.550064 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:12.550069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:12.550701 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:12.550904 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.550913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.551959 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.551962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.553000 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:12.553014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.553836 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.553841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:12.554041 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.554705 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:12.554727 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:12.558658 (+ 3931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":334,"threads_started":1}
W20260810 11:45:12.559420 30895 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:12.548818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:33795 (local address 127.20.93.190:46325)
0810 11:45:12.549036 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:12.549044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:12.549497 (+ 453us) server_negotiation.cc:407] Connection header received
0810 11:45:12.549781 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:12.549787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:12.549893 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:12.550015 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:12.551043 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.551816 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.553145 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:12.553665 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:12.553778 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:12.558932 (+ 5154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:33795: BlockingRecv error: recv got EOF from 127.20.93.129:33795 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:13.181633 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:13.171671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:49075)
0810 11:45:13.172513 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:13.172530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:13.172693 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:13.173147 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:13.173152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:13.173741 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:13.173963 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.173976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.175143 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.175148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.176083 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.176094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.176828 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.176833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.176959 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.177644 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:13.177676 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:13.181274 (+ 3598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":588,"threads_started":1}
W20260810 11:45:13.181860 30899 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:13.172067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:49075 (local address 127.20.93.190:46325)
0810 11:45:13.172636 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:13.172642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:13.172663 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:45:13.172836 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:13.172840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:13.172951 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:13.173074 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:13.174141 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.175007 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.176245 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.176695 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.176795 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.181531 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:49075: BlockingRecv error: recv got EOF from 127.20.93.130:49075 (error 108)
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":281,"threads_started":1}
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:13.545506 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:13.534735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:44989)
0810 11:45:13.535357 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:13.535377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:13.535511 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:13.535855 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:13.535860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:13.536412 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:13.536609 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.536621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.537733 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.537741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.539249 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.539265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.540075 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.540080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.540274 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.540971 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:13.540996 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:13.545168 (+ 4172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":355,"threads_started":1}
W20260810 11:45:13.545778 30899 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:13.534793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:44989 (local address 127.20.93.190:46325)
0810 11:45:13.535030 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:13.535040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:13.535461 (+ 421us) server_negotiation.cc:407] Connection header received
0810 11:45:13.535635 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:13.535639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:13.535702 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:13.535787 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:13.536759 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.537499 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.539423 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.539915 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.540021 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.545458 (+ 5437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:44989: BlockingRecv error: recv got EOF from 127.20.93.131:44989 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:13 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:13.569308 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:13.560029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:47679)
0810 11:45:13.560590 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:13.560605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:13.560780 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:13.561161 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:13.561166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:13.561688 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:13.561855 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.561864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.562898 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.562909 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.563850 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:13.563862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.564621 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.564626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:13.564806 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.565462 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:13.565481 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:13.568967 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":305,"threads_started":1}
W20260810 11:45:13.569597 30899 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:13.560095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:47679 (local address 127.20.93.190:46325)
0810 11:45:13.560367 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:13.560376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:13.560705 (+ 329us) server_negotiation.cc:407] Connection header received
0810 11:45:13.560925 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:13.560929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:13.561010 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:13.561120 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:13.561997 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.562748 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.564007 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:13.564440 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:13.564594 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:13.569225 (+ 4631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:47679: BlockingRecv error: recv got EOF from 127.20.93.129:47679 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:14.192487 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:14.182663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:56965)
0810 11:45:14.183262 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:14.183281 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:14.183408 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:14.183773 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:14.183777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:14.184169 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:14.184327 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.184336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.185322 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.185327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.186508 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.186546 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.187337 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.187342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.187469 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.188056 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:14.188073 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:14.192146 (+ 4073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":366,"threads_started":1}
W20260810 11:45:14.192696 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:14.182757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:56965 (local address 127.20.93.190:46325)
0810 11:45:14.183269 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:14.183274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:14.183360 (+ 86us) server_negotiation.cc:407] Connection header received
0810 11:45:14.183531 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:14.183534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:14.183621 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:14.183709 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:14.184446 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.185142 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.186663 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.187178 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.187279 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.192394 (+ 5115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:56965: BlockingRecv error: recv got EOF from 127.20.93.130:56965 (error 108)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":285,"threads_started":1}
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:14.556414 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:14.546783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:35727)
0810 11:45:14.547327 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:14.547350 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:14.547527 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:14.547860 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:14.547863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:14.548289 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:14.548455 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.548464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.549535 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.549539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.550697 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.550709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.551658 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.551667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.551827 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.552555 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:14.552574 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:14.556159 (+ 3585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"spinlock_wait_cycles":28928,"thread_start_us":409,"threads_started":1}
W20260810 11:45:14.556666 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:14.546968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35727 (local address 127.20.93.190:46325)
0810 11:45:14.547182 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:14.547190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:14.547462 (+ 272us) server_negotiation.cc:407] Connection header received
0810 11:45:14.547647 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:14.547652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:14.547719 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:14.547816 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:14.548639 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.549399 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.550926 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.551478 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.551594 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.556425 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:35727: BlockingRecv error: recv got EOF from 127.20.93.131:35727 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:14 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:14.581096 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:14.570519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:43921)
0810 11:45:14.571315 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:14.571332 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:14.571514 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:14.571889 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:14.571895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:14.572359 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:14.572545 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.572554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.574094 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.574099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.575303 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:14.575317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.576278 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.576283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:14.576399 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.577166 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:14.577185 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:14.580816 (+ 3631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":376,"threads_started":1}
W20260810 11:45:14.581295 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:14.570661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:43921 (local address 127.20.93.190:46325)
0810 11:45:14.571043 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:14.571052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:14.571436 (+ 384us) server_negotiation.cc:407] Connection header received
0810 11:45:14.571661 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:14.571665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:14.571746 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:14.571856 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:14.572702 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.573930 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.575467 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:14.576126 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:14.576288 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:14.581043 (+ 4755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:43921: BlockingRecv error: recv got EOF from 127.20.93.129:43921 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:15.203433 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:15.193728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:32789)
0810 11:45:15.194814 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:15.194831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:15.194988 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:15.195345 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:15.195348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:15.195800 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:15.195990 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.195998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.197057 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.197061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.198198 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.198210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.198890 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.198894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.199002 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.199664 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:15.199682 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:15.203182 (+ 3500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":729,"threads_started":1}
W20260810 11:45:15.203737 30907 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:15.193725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:32789 (local address 127.20.93.190:46325)
0810 11:45:15.194390 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:15.194394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:15.194921 (+ 527us) server_negotiation.cc:407] Connection header received
0810 11:45:15.195130 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:15.195141 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:15.195212 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:15.195300 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:15.196124 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.196934 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.198323 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.198751 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.198857 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.203414 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:32789: BlockingRecv error: recv got EOF from 127.20.93.130:32789 (error 108)
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":413,"threads_started":1}
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:15.570708 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:15.557466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:45343)
0810 11:45:15.558109 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:15.558126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:15.558294 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:15.558663 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:15.558668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:15.559159 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:15.559336 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.559346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.560496 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.560503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.561485 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.561497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.562326 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.562331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.562517 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.566490 (+ 3973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:15.566515 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:15.570405 (+ 3890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":281,"threads_started":1}
W20260810 11:45:15.570837 30907 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:15.557510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:45343 (local address 127.20.93.190:46325)
0810 11:45:15.557752 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:15.557761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:15.558226 (+ 465us) server_negotiation.cc:407] Connection header received
0810 11:45:15.558427 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:15.558433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:15.558514 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:15.558629 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:15.559498 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.560331 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.561712 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.562163 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.562300 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.570608 (+ 8308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:45343: BlockingRecv error: recv got EOF from 127.20.93.131:45343 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:15 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:15.591279 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:15.582191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:55591)
0810 11:45:15.582722 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:15.582738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:15.582848 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:15.583146 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:15.583149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:15.583553 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:15.583736 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.583744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.584781 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.584786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.585868 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:15.585881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.586604 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.586608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:15.586771 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.587492 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:15.587509 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:15.591027 (+ 3518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":267,"threads_started":1}
W20260810 11:45:15.591557 30907 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:15.582359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:55591 (local address 127.20.93.190:46325)
0810 11:45:15.582560 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:15.582565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:15.582806 (+ 241us) server_negotiation.cc:407] Connection header received
0810 11:45:15.582944 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:15.582947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:15.583026 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:15.583109 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:15.583878 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.584643 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.585988 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:15.586452 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:15.586556 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:15.591256 (+ 4700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:55591: BlockingRecv error: recv got EOF from 127.20.93.129:55591 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:16.214483 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:16.204427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:46989)
0810 11:45:16.204945 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:16.204961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:16.205069 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:16.205675 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:16.205678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:16.206190 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:16.206367 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.206376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.207644 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.207648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.208582 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.208593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.209548 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.209553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.209929 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.210679 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:16.210705 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:16.214220 (+ 3515us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":270,"threads_started":1}
W20260810 11:45:16.214708 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:16.204538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:46989 (local address 127.20.93.190:46325)
0810 11:45:16.205312 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:16.205317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:16.205334 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:45:16.205384 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:16.205388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:16.205461 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:16.205653 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:16.206520 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.207512 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.208707 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.209177 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.209306 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.214439 (+ 5133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:46989: BlockingRecv error: recv got EOF from 127.20.93.130:46989 (error 108)
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":531,"threads_started":1}
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:16.581840 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:16.571833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:50859)
0810 11:45:16.572426 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:16.572442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:16.572560 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:16.572816 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:16.572819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:16.573330 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:16.573500 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.573511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.574964 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.574967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.575913 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.575925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.576723 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.576727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.576839 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.577504 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:16.577523 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:16.581480 (+ 3957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":357,"threads_started":1}
W20260810 11:45:16.582007 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:16.571947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:50859 (local address 127.20.93.190:46325)
0810 11:45:16.572322 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:16.572330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:16.572526 (+ 196us) server_negotiation.cc:407] Connection header received
0810 11:45:16.572649 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:16.572653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:16.572720 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:16.572802 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:16.573678 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.574829 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.576071 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.576550 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.576651 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.581742 (+ 5091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:50859: BlockingRecv error: recv got EOF from 127.20.93.131:50859 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:16 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:16.602237 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:16.592508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:52425)
0810 11:45:16.593104 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:16.593122 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:16.593314 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:16.593805 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:16.593810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:16.594315 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:16.594530 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.594544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.595582 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.595588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.596634 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:16.596648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.597403 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.597411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:16.597658 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.598264 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:16.598281 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:16.601893 (+ 3612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":331,"threads_started":1}
W20260810 11:45:16.602571 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:16.592508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:52425 (local address 127.20.93.190:46325)
0810 11:45:16.592701 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:16.592709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:16.593223 (+ 514us) server_negotiation.cc:407] Connection header received
0810 11:45:16.593468 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:16.593473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:16.593618 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:16.593745 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:16.594704 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.595432 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.596778 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:16.597202 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:16.597348 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:16.602119 (+ 4771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:52425: BlockingRecv error: recv got EOF from 127.20.93.129:52425 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:17.225081 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:17.215529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:56659)
0810 11:45:17.216113 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:17.216132 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:17.216264 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:17.216612 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:17.216616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:17.217052 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:17.217204 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.217213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.218228 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.218232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.219202 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.219214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.219951 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.219955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.220089 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.220736 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:17.220755 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:17.224719 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":290,"threads_started":1}
W20260810 11:45:17.225301 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:17.215568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:56659 (local address 127.20.93.190:46325)
0810 11:45:17.216122 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:17.216127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:17.216213 (+ 86us) server_negotiation.cc:407] Connection header received
0810 11:45:17.216392 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:17.216396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:17.216468 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:17.216560 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:17.217343 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.218094 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.219349 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.219804 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.219902 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.225021 (+ 5119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:56659: BlockingRecv error: recv got EOF from 127.20.93.130:56659 (error 108)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":331,"threads_started":1}
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:17.592371 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:17.583141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:45629)
0810 11:45:17.583663 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:17.583684 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:17.583805 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:17.584148 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:17.584151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:17.584557 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:17.584707 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.584716 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.585851 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.585855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.586924 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.586938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.587677 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.587683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.587868 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.588428 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:17.588446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:17.591974 (+ 3528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":275,"threads_started":1}
W20260810 11:45:17.592463 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:17.583225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:45629 (local address 127.20.93.190:46325)
0810 11:45:17.583475 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:17.583481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:17.583773 (+ 292us) server_negotiation.cc:407] Connection header received
0810 11:45:17.583911 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:17.583916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:17.583981 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:17.584066 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:17.584838 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.585700 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.587081 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.587514 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.587665 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.592209 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:45629: BlockingRecv error: recv got EOF from 127.20.93.131:45629 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:17 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:17.612680 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:17.603234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:46925)
0810 11:45:17.603791 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:17.603816 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:17.603931 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:17.604289 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:17.604294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:17.604769 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:17.604921 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.604930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.605995 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.605999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.607056 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:17.607068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.607763 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.607767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:17.607880 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.608538 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:17.608556 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:17.612440 (+ 3884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":327,"threads_started":1}
W20260810 11:45:17.612906 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:17.603278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46925 (local address 127.20.93.190:46325)
0810 11:45:17.603444 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:17.603449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:17.603893 (+ 444us) server_negotiation.cc:407] Connection header received
0810 11:45:17.604027 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:17.604031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:17.604110 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:17.604195 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:17.605070 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.605879 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.607182 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:17.607622 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:17.607723 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:17.612622 (+ 4899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:46925: BlockingRecv error: recv got EOF from 127.20.93.129:46925 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:18.235596 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:18.226339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:44251)
0810 11:45:18.226811 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:18.226828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:18.226931 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:18.227391 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:18.227394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:18.227743 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:18.227895 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.227902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.228846 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.228850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.229955 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.229968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.230634 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.230638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.230748 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.231401 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:18.231419 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:18.235243 (+ 3824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
W20260810 11:45:18.235848 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:18.226404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:44251 (local address 127.20.93.190:46325)
0810 11:45:18.227134 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:18.227138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:18.227153 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:45:18.227194 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:18.227198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:18.227262 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:18.227347 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:18.228025 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.228730 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.230100 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.230502 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.230602 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.235528 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:44251: BlockingRecv error: recv got EOF from 127.20.93.130:44251 (error 108)
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":528,"threads_started":1}
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:18.602779 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:18.593521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:53741)
0810 11:45:18.594239 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:18.594255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:18.594368 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:18.594677 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:18.594680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:18.595038 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:18.595204 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.595224 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.596206 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.596209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.597193 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.597204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.598032 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.598037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.598148 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.598795 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:18.598812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:18.602476 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":455,"threads_started":1}
W20260810 11:45:18.602865 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:18.593707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:53741 (local address 127.20.93.190:46325)
0810 11:45:18.593953 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:18.593958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:18.594337 (+ 379us) server_negotiation.cc:407] Connection header received
0810 11:45:18.594467 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:18.594471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:18.594546 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:18.594630 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:18.595341 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.596088 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.597334 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.597902 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.597997 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.602672 (+ 4675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:53741: BlockingRecv error: recv got EOF from 127.20.93.131:53741 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:18 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:18.622501 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:18.613762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:36993)
0810 11:45:18.614405 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:18.614427 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:18.614586 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:18.614946 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:18.614950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:18.615362 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:18.615539 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.615548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.616511 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.616514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.617412 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:18.617422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.618164 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.618168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:18.618297 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.618969 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:18.618986 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:18.622277 (+ 3291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":376,"threads_started":1}
W20260810 11:45:18.622779 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:18.613798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:36993 (local address 127.20.93.190:46325)
0810 11:45:18.614025 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:18.614032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:18.614522 (+ 490us) server_negotiation.cc:407] Connection header received
0810 11:45:18.614743 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:18.614746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:18.614820 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:18.614900 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:18.615667 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.616391 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.617616 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:18.618032 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:18.618128 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:18.622462 (+ 4334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:36993: BlockingRecv error: recv got EOF from 127.20.93.129:36993 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:19.245703 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:19.236608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:35741)
0810 11:45:19.237077 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:19.237094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:19.237204 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:19.237754 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:19.237758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:19.238200 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:19.238411 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.238421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.239452 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.239456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.240368 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.240380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.241054 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.241058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.241188 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.241830 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:19.241849 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:19.245414 (+ 3565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":245,"threads_started":1}
W20260810 11:45:19.245879 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:19.236757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:35741 (local address 127.20.93.190:46325)
0810 11:45:19.237417 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:19.237422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:19.237437 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:45:19.237485 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:19.237488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:19.237553 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:19.237661 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:19.238561 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.239316 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.240498 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.240908 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.241004 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.245630 (+ 4626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:35741: BlockingRecv error: recv got EOF from 127.20.93.130:35741 (error 108)
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":431,"threads_started":1}
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:19.613108 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:19.603690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:46635)
0810 11:45:19.604224 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:19.604242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:19.604361 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:19.604686 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:19.604690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:19.605289 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:19.605555 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.605568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.607020 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.607025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.608043 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.608055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.608861 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.608865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.608984 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.609594 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:19.609612 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:19.612885 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":301,"threads_started":1}
W20260810 11:45:19.613263 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:19.603774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:46635 (local address 127.20.93.190:46325)
0810 11:45:19.603940 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:19.603945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:19.604320 (+ 375us) server_negotiation.cc:407] Connection header received
0810 11:45:19.604461 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:19.604465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:19.604526 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:19.604609 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:19.605769 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.606900 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.608163 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.608723 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.608828 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.613075 (+ 4247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:46635: BlockingRecv error: recv got EOF from 127.20.93.131:46635 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:19 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:19.632967 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:19.623428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:60353)
0810 11:45:19.623918 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:19.623934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:19.624085 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:19.624424 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:19.624429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:19.624837 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:19.624996 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.625004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.626073 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.626086 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.627027 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:19.627038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.627821 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.627827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:19.628004 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.628655 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:19.628677 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:19.632606 (+ 3929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":275,"threads_started":1}
W20260810 11:45:19.633081 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:19.623472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:60353 (local address 127.20.93.190:46325)
0810 11:45:19.623680 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:19.623686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:19.624016 (+ 330us) server_negotiation.cc:407] Connection header received
0810 11:45:19.624202 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:19.624207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:19.624291 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:19.624394 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:19.625136 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.625939 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.627192 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:19.627636 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:19.627796 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:19.632821 (+ 5025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:60353: BlockingRecv error: recv got EOF from 127.20.93.129:60353 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:20.258395 30928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:20.246827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:48419)
0810 11:45:20.247335 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:20.247361 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:20.247505 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:20.248047 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:20.248055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:20.248651 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:20.248892 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.248906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.250409 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.250413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.251665 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.251680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.252663 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.252669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.252891 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.253668 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:20.253693 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:20.258048 (+ 4355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
W20260810 11:45:20.258913 30929 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:20.247183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:48419 (local address 127.20.93.190:46325)
0810 11:45:20.247629 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:20.247638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:20.247657 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:45:20.247725 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:20.247739 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:20.247854 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:20.247975 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:20.249060 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.250274 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.251825 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.252474 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.252605 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.258397 (+ 5792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:48419: BlockingRecv error: recv got EOF from 127.20.93.130:48419 (error 108)
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":339,"threads_started":1}
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:20.628089 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:20.614219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:36437)
0810 11:45:20.614777 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:20.614794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:20.614955 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:20.615318 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:20.615323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:20.615736 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:20.615917 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.615927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.616951 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.616958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.617954 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.617986 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.618748 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.618754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.618937 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.623996 (+ 5059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:20.624016 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:20.627710 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":308,"threads_started":1}
W20260810 11:45:20.628176 30929 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:20.614304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:36437 (local address 127.20.93.190:46325)
0810 11:45:20.614555 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:20.614563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:20.614882 (+ 319us) server_negotiation.cc:407] Connection header received
0810 11:45:20.615090 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:20.615096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:20.615173 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:20.615277 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:20.616062 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.616808 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.618124 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.618531 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.618696 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.627924 (+ 9228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:36437: BlockingRecv error: recv got EOF from 127.20.93.131:36437 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:20 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:20.643153 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:20.634020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:42645)
0810 11:45:20.634589 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:20.634606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:20.634780 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:20.635210 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:20.635214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:20.635683 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:20.635836 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.635845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.636827 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.636831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.637800 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:20.637812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.638462 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.638466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:20.638599 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.639305 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:20.639324 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:20.642889 (+ 3565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":320,"threads_started":1}
W20260810 11:45:20.643388 30929 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:20.634120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:42645 (local address 127.20.93.190:46325)
0810 11:45:20.634363 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:20.634370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:20.634693 (+ 323us) server_negotiation.cc:407] Connection header received
0810 11:45:20.634949 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:20.634954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:20.635034 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:20.635130 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:20.635965 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.636690 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.637934 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:20.638334 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:20.638431 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:20.643112 (+ 4681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:42645: BlockingRecv error: recv got EOF from 127.20.93.129:42645 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.130@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:21.268404 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:21.259498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.130:44381)
0810 11:45:21.260053 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:21.260070 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:21.260176 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:21.260538 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:21.260541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:21.260947 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:21.261098 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.261106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.262324 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.262328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.263262 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.263274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.263977 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.263981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.264102 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.264777 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:21.264795 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:21.268168 (+ 3373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":322,"threads_started":1}
W20260810 11:45:21.268652 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:21.259680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:44381 (local address 127.20.93.190:46325)
0810 11:45:21.260238 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:21.260243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:21.260260 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:45:21.260307 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:21.260310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:21.260389 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:21.260483 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:21.261288 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.262185 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.263406 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.263849 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.263974 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.268372 (+ 4398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.130:44381: BlockingRecv error: recv got EOF from 127.20.93.130:44381 (error 108)
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":323,"threads_started":1}
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.131@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:21.640694 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:21.629294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.131:55707)
0810 11:45:21.629856 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:21.629874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:21.629987 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:21.630454 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:21.630458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:21.630929 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:21.631085 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.631095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.633022 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.633027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.634466 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.634485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.635618 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.635624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.635835 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.636494 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:21.636512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:21.640316 (+ 3804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":342,"threads_started":1}
W20260810 11:45:21.640789 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:21.629394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:55707 (local address 127.20.93.190:46325)
0810 11:45:21.629681 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:21.629689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:21.629959 (+ 270us) server_negotiation.cc:407] Connection header received
0810 11:45:21.630108 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:21.630117 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:21.630223 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:21.630356 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:21.631246 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.632694 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.634654 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.635469 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.635605 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.640534 (+ 4929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.131:55707: BlockingRecv error: recv got EOF from 127.20.93.131:55707 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:45:21 dist-test-slave-njxd krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.93.129@KRBTEST.COM for oryx/127.20.93.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:45:21.653968 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:21.644422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:46325 (local address 127.20.93.129:46937)
0810 11:45:21.645015 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:21.645031 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:21.645150 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:21.645544 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:21.645549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:21.646034 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:21.646300 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.646316 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.647404 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.647409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.648370 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:21.648381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.649085 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.649091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:21.649281 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.649978 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:21.649995 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:21.653543 (+ 3548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.93.190:46325: Server oryx/127.20.93.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":305,"threads_started":1}
W20260810 11:45:21.654241 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:21.644522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46937 (local address 127.20.93.190:46325)
0810 11:45:21.644883 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:21.644888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:21.645116 (+ 228us) server_negotiation.cc:407] Connection header received
0810 11:45:21.645309 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:21.645314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:21.645382 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:21.645477 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:21.646465 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.647255 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.648516 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:21.648936 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:21.649071 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:21.653812 (+ 4741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.93.129:46937: BlockingRecv error: recv got EOF from 127.20.93.129:46937 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
I20260810 11:45:22.149214 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30419
I20260810 11:45:22.163168 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:22.150495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.20.93.129:37435)
0810 11:45:22.151202 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:22.151210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:22.151245 (+ 35us) server_negotiation.cc:407] Connection header received
0810 11:45:22.151312 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:22.151318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:22.151589 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:22.151769 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:22.153617 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.155210 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.156721 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.157251 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.157352 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:22.159925 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:22.159933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:22.159948 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:22.159987 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:22.162643 (+ 2656us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:45:22.162947 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53210: Request ticket server kudu/127.20.93.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":371,"threads_started":1}
W20260810 11:45:22.163355 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30419: Not authorized: Client connection negotiation failed: client connection to 127.20.93.129:37435: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:45:22.163604 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30419
I20260810 11:45:22.184369 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30585
I20260810 11:45:22.198560 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:22.185627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.20.93.130:44543)
0810 11:45:22.186280 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:22.186289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:22.186317 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:45:22.186399 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:22.186407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:22.186669 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:22.187020 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:22.188490 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.189569 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.191065 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.191660 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.191816 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:22.194874 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:22.194883 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:22.194899 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:22.194942 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:22.197710 (+ 2768us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:45:22.198141 (+ 431us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43826: Request ticket server kudu/127.20.93.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":330,"threads_started":1}
W20260810 11:45:22.198582 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30585: Not authorized: Client connection negotiation failed: client connection to 127.20.93.130:44543: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 11:45:22.198711 30940 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43826: Request ticket server kudu/127.20.93.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:45:22.198760 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30585
I20260810 11:45:22.220145 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30733
I20260810 11:45:22.234759 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:22.221466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.20.93.131:35069)
0810 11:45:22.222220 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:22.222227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:22.222257 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:45:22.222326 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:22.222333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:22.222591 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:22.222779 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:22.224924 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.226312 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.227586 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.228202 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.228321 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:22.231203 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:22.231214 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:22.231230 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:22.231278 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:22.234060 (+ 2782us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:45:22.234461 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57788: Request ticket server kudu/127.20.93.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":430,"threads_started":1}
W20260810 11:45:22.234896 30941 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57788: Request ticket server kudu/127.20.93.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 11:45:22.234927 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30733: Not authorized: Client connection negotiation failed: client connection to 127.20.93.131:35069: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.93.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:45:22.235138 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30733
I20260810 11:45:22.256017 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30345
I20260810 11:45:22.257462 30411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:22.383863 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30345
2026-08-10T11:45:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21782 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T11:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:22Z Disabled control of system clock
I20260810 11:45:22.483498 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38411
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 11:45:22.856887 30946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:22.857244 30946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:22.857331 30946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:22.865168 30946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:22.865315 30946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:22.865361 30946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:22.865413 30946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:22.865459 30946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:22.875699 30946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:22.877130 30946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:22.879062 30946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:22.893092 30952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:22.893181 30955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:22.893221 30953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:22.893896 30946 server_base.cc:1034] running on GCE node
I20260810 11:45:22.895056 30946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:22.896715 30946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:22.898083 30946 hybrid_clock.cc:648] HybridClock initialized: now 1786362322898028 us; error 50 us; skew 500 ppm
I20260810 11:45:22.898584 30946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:22.901438 30946 webserver.cc:460] Webserver started at http://127.20.93.190:43067/ using document root <none> and password file <none>
I20260810 11:45:22.902345 30946 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:22.902525 30946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:22.902918 30946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:22.906529 30946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "57b85b98ffc348eab3052bf897b1c6d8"
format_stamp: "Formatted at 2026-08-10 11:45:22 on dist-test-slave-njxd"
server_key: "9d344267c5bee46a5912879827f05a23"
server_key_iv: "5d4da88a7826a983178bcbf60fa44546"
server_key_version: "encryptionkey@0"
I20260810 11:45:22.907533 30946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "57b85b98ffc348eab3052bf897b1c6d8"
format_stamp: "Formatted at 2026-08-10 11:45:22 on dist-test-slave-njxd"
server_key: "9d344267c5bee46a5912879827f05a23"
server_key_iv: "5d4da88a7826a983178bcbf60fa44546"
server_key_version: "encryptionkey@0"
I20260810 11:45:22.914026 30946 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:45:22.918452 30962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:22.920249 30946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:45:22.920524 30946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "57b85b98ffc348eab3052bf897b1c6d8"
format_stamp: "Formatted at 2026-08-10 11:45:22 on dist-test-slave-njxd"
server_key: "9d344267c5bee46a5912879827f05a23"
server_key_iv: "5d4da88a7826a983178bcbf60fa44546"
server_key_version: "encryptionkey@0"
I20260810 11:45:22.920815 30946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:22.947894 30946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:22.948939 30946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:22.949275 30946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:22.969558 30946 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38411
I20260810 11:45:22.969653 31017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38411 every 8 connection(s)
I20260810 11:45:22.971218 30946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:45:22.975447 31018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:22.976131 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 30946
I20260810 11:45:22.976353 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:45:22.977041 20854 external_mini_cluster.cc:1428] Setting key b71e684def94ce407338adb20dda7009
I20260810 11:45:22.989940 31018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8: Bootstrap starting.
I20260810 11:45:22.992400 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:22.979527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.20.93.190:38411)
0810 11:45:22.980672 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:22.980685 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:22.980744 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:45:22.982579 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:22.982608 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:22.983225 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:22.983686 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:22.984447 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.985892 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.987350 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:22.988156 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:22.988381 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:22.989778 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:22.989786 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:22.989802 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:22.990448 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:22.990541 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:22.990564 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:22.990699 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:22.990959 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 11:45:22.991333 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":384,"threads_started":1}
I20260810 11:45:22.994017 31018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:22.995237 31018 log.cc:824] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:22.998509 31018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8: No bootstrap required, opened a new log
I20260810 11:45:23.007113 31018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } }
I20260810 11:45:23.007725 31018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:23.007867 31018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b85b98ffc348eab3052bf897b1c6d8, State: Initialized, Role: FOLLOWER
I20260810 11:45:23.008706 31018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } }
I20260810 11:45:23.008947 31018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:45:23.009068 31018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:45:23.009222 31018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:23.011605 31018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } }
I20260810 11:45:23.012151 31018 leader_election.cc:302] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57b85b98ffc348eab3052bf897b1c6d8; no voters:
I20260810 11:45:23.012781 31018 leader_election.cc:288] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:45:23.012964 31023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:45:23.013892 31023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 57b85b98ffc348eab3052bf897b1c6d8, State: Running, Role: LEADER
I20260810 11:45:23.014559 31023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } }
I20260810 11:45:23.015421 31018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:45:23.018317 31027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57b85b98ffc348eab3052bf897b1c6d8. Latest consensus state: current_term: 1 leader_uuid: "57b85b98ffc348eab3052bf897b1c6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } } }
I20260810 11:45:23.018906 31027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:23.019310 31025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57b85b98ffc348eab3052bf897b1c6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b85b98ffc348eab3052bf897b1c6d8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38411 } } }
I20260810 11:45:23.019826 31025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:23.022845 31033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:45:23.028611 31033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:45:23.041817 31033 catalog_manager.cc:1347] Generated new cluster ID: 5255d8fef2844f9ba423dd4f8b66c802
I20260810 11:45:23.041972 31033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:45:23.057516 31033 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:45:23.058987 31033 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:45:23.068617 31033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57b85b98ffc348eab3052bf897b1c6d8: Generated new TSK 0
I20260810 11:45:23.069906 31033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:45:23.083951 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--builtin_ntp_servers=127.20.93.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 11:45:23.459843 31045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:23.460136 31045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:23.460294 31045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:23.467738 31045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:23.467859 31045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:23.468034 31045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:45:23.477473 31045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:23.478896 31045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:23.480791 31045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:23.494460 31053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:23.494549 31052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:23.494488 31055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:23.499105 31045 server_base.cc:1034] running on GCE node
I20260810 11:45:23.499939 31045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:23.500985 31045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:23.502223 31045 hybrid_clock.cc:648] HybridClock initialized: now 1786362323502158 us; error 62 us; skew 500 ppm
I20260810 11:45:23.502575 31045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:23.505048 31045 webserver.cc:460] Webserver started at http://127.20.93.129:43103/ using document root <none> and password file <none>
I20260810 11:45:23.505903 31045 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:23.506042 31045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:23.506385 31045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:23.509377 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "48bb443768e3495783f69faf636f48bb"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "0b2cd9d2911f7a8cdc5e989ba14ef67e"
server_key_iv: "3727278035dbfa988e684a4ce5a6ef93"
server_key_version: "encryptionkey@0"
I20260810 11:45:23.510324 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "48bb443768e3495783f69faf636f48bb"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "0b2cd9d2911f7a8cdc5e989ba14ef67e"
server_key_iv: "3727278035dbfa988e684a4ce5a6ef93"
server_key_version: "encryptionkey@0"
I20260810 11:45:23.515722 31045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 11:45:23.519488 31061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:23.521092 31045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:45:23.521277 31045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "48bb443768e3495783f69faf636f48bb"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "0b2cd9d2911f7a8cdc5e989ba14ef67e"
server_key_iv: "3727278035dbfa988e684a4ce5a6ef93"
server_key_version: "encryptionkey@0"
I20260810 11:45:23.521519 31045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:23.532996 31045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:23.533982 31045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:23.534299 31045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:23.535283 31045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:23.536792 31045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:23.536945 31045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:23.537031 31045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:23.537091 31045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:23.572527 31045 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:36491
I20260810 11:45:23.572620 31179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:36491 every 8 connection(s)
I20260810 11:45:23.574083 31045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:45:23.577001 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31045
I20260810 11:45:23.577199 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:45:23.578068 20854 external_mini_cluster.cc:1428] Setting key 2106f3f8bb3550a6f674b2b18b64dc54
I20260810 11:45:23.581379 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--builtin_ntp_servers=127.20.93.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 11:45:23.589098 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:23.576862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:45053 (local address 127.20.93.190:38411)
0810 11:45:23.577345 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:23.577350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:23.579358 (+ 2008us) server_negotiation.cc:407] Connection header received
0810 11:45:23.581157 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:23.581165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:23.581260 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:23.581397 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:23.582637 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:23.583991 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:23.585695 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:23.586410 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:23.586583 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:23.588036 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:23.588045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:23.588052 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:23.588081 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:23.588138 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:23.588159 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:23.588255 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:23.588490 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 11:45:23.588630 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":279,"threads_started":1}
I20260810 11:45:23.590123 31183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:23.577696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38411 (local address 127.20.93.129:45053)
0810 11:45:23.578856 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:23.578915 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:23.580852 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:23.581727 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:23.581735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:23.581842 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:23.582444 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:23.582460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:23.584164 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:23.584167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:23.585496 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:23.585508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:23.586659 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:23.586665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:23.586877 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:23.587652 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:23.587655 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:23.587702 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:23.587793 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:23.588258 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:23.588266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:23.588272 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:45:23.588423 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 11:45:23.588789 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":368,"threads_started":1}
I20260810 11:45:23.591933 31180 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38411
I20260810 11:45:23.592293 31180 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:23.593107 31180 heartbeater.cc:511] Master 127.20.93.190:38411 requested a full tablet report, sending...
I20260810 11:45:23.595371 30980 ts_manager.cc:194] Registered new tserver with Master: 48bb443768e3495783f69faf636f48bb (127.20.93.129:36491)
I20260810 11:45:23.596901 30980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.129:45053
W20260810 11:45:23.942452 31184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:23.942808 31184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:23.942968 31184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:23.950341 31184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:23.950481 31184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:23.950672 31184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:45:23.960245 31184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:23.961696 31184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:23.963632 31184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:23.977435 31189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:23.977676 31192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:23.978176 31190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:23.979732 31184 server_base.cc:1034] running on GCE node
I20260810 11:45:23.980345 31184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:23.981359 31184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:23.982597 31184 hybrid_clock.cc:648] HybridClock initialized: now 1786362323982553 us; error 35 us; skew 500 ppm
I20260810 11:45:23.982939 31184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:23.985278 31184 webserver.cc:460] Webserver started at http://127.20.93.130:35793/ using document root <none> and password file <none>
I20260810 11:45:23.986053 31184 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:23.986191 31184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:23.986517 31184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:23.989423 31184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "9082ad743f264ca2811fadf91fd1a9a0"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "4359f72fbba1353a086742824c9d03d1"
server_key_iv: "9f8f432a67798a4bff715678416139f1"
server_key_version: "encryptionkey@0"
I20260810 11:45:23.990339 31184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "9082ad743f264ca2811fadf91fd1a9a0"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "4359f72fbba1353a086742824c9d03d1"
server_key_iv: "9f8f432a67798a4bff715678416139f1"
server_key_version: "encryptionkey@0"
I20260810 11:45:23.995695 31184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:45:23.999171 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.000525 31184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 11:45:24.000723 31184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "9082ad743f264ca2811fadf91fd1a9a0"
format_stamp: "Formatted at 2026-08-10 11:45:23 on dist-test-slave-njxd"
server_key: "4359f72fbba1353a086742824c9d03d1"
server_key_iv: "9f8f432a67798a4bff715678416139f1"
server_key_version: "encryptionkey@0"
I20260810 11:45:24.000957 31184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:24.031170 31184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:24.032087 31184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:24.032404 31184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:24.033493 31184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:24.035192 31184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:24.035315 31184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.035462 31184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:24.035552 31184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.070020 31184 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:46291
I20260810 11:45:24.070127 31323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:46291 every 8 connection(s)
I20260810 11:45:24.071370 31184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:45:24.073998 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31184
I20260810 11:45:24.074187 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:45:24.074658 20854 external_mini_cluster.cc:1428] Setting key 6973dd05918b1f10224d68a866b729fb
I20260810 11:45:24.077703 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--builtin_ntp_servers=127.20.93.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 11:45:24.084790 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.074065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:41703 (local address 127.20.93.190:38411)
0810 11:45:24.074282 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.074288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.076009 (+ 1721us) server_negotiation.cc:407] Connection header received
0810 11:45:24.077889 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.077895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.077988 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.078116 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.079291 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.080023 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.081739 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.082176 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.082318 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.083903 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:24.083908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:24.083912 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:24.083933 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:24.083967 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:24.083980 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:24.084091 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:24.084340 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:45:24.084450 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:45:24.086061 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.074663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38411 (local address 127.20.93.130:41703)
0810 11:45:24.075832 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:24.075887 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:24.077531 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:24.078384 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:24.078393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:24.078491 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:24.079098 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:24.079115 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.080197 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.080208 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:24.081521 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:24.081534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.082425 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.082433 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:24.082692 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.083499 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:24.083502 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:24.083546 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:24.083660 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:24.084099 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:24.084113 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:24.084124 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:45:24.084327 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 11:45:24.084794 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":374,"threads_started":1}
I20260810 11:45:24.087749 31325 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38411
I20260810 11:45:24.088073 31325 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:24.089146 31325 heartbeater.cc:511] Master 127.20.93.190:38411 requested a full tablet report, sending...
I20260810 11:45:24.090873 30980 ts_manager.cc:194] Registered new tserver with Master: 9082ad743f264ca2811fadf91fd1a9a0 (127.20.93.130:46291)
I20260810 11:45:24.091708 30980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.130:41703
W20260810 11:45:24.421923 31328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:24.422303 31328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:24.422451 31328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:24.430244 31328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:24.430387 31328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:24.430564 31328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:45:24.439852 31328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:24.441257 31328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:24.443183 31328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:24.457121 31333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:24.457221 31334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:24.457202 31336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:24.457646 31328 server_base.cc:1034] running on GCE node
I20260810 11:45:24.458503 31328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:24.459470 31328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:24.460704 31328 hybrid_clock.cc:648] HybridClock initialized: now 1786362324460628 us; error 41 us; skew 500 ppm
I20260810 11:45:24.461002 31328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:24.463516 31328 webserver.cc:460] Webserver started at http://127.20.93.131:33265/ using document root <none> and password file <none>
I20260810 11:45:24.464205 31328 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:24.464301 31328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:24.464568 31328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:24.467537 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "3fe14343fff943b48ad9390ee97e97cd"
format_stamp: "Formatted at 2026-08-10 11:45:24 on dist-test-slave-njxd"
server_key: "aec3d71b5456f9c0d21823c97e5afb78"
server_key_iv: "1a562f7a1cefe7e1918de0d2ecf55262"
server_key_version: "encryptionkey@0"
I20260810 11:45:24.468302 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "3fe14343fff943b48ad9390ee97e97cd"
format_stamp: "Formatted at 2026-08-10 11:45:24 on dist-test-slave-njxd"
server_key: "aec3d71b5456f9c0d21823c97e5afb78"
server_key_iv: "1a562f7a1cefe7e1918de0d2ecf55262"
server_key_version: "encryptionkey@0"
I20260810 11:45:24.473899 31328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 11:45:24.477578 31343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.479051 31328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:45:24.479195 31328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "3fe14343fff943b48ad9390ee97e97cd"
format_stamp: "Formatted at 2026-08-10 11:45:24 on dist-test-slave-njxd"
server_key: "aec3d71b5456f9c0d21823c97e5afb78"
server_key_iv: "1a562f7a1cefe7e1918de0d2ecf55262"
server_key_version: "encryptionkey@0"
I20260810 11:45:24.479389 31328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:24.504731 31328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:24.505767 31328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:24.506099 31328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:24.507198 31328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:24.508834 31328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:24.508934 31328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.509092 31328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:24.509176 31328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:24.544219 31328 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:38763
I20260810 11:45:24.544318 31467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:38763 every 8 connection(s)
I20260810 11:45:24.545691 31328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:45:24.549422 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31328
I20260810 11:45:24.549682 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:45:24.550267 20854 external_mini_cluster.cc:1428] Setting key 84e9fd317e7cd3eaf83209e35470d152
I20260810 11:45:24.559393 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.548372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:38515 (local address 127.20.93.190:38411)
0810 11:45:24.548609 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.548616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.550450 (+ 1834us) server_negotiation.cc:407] Connection header received
0810 11:45:24.552661 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.552667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.552763 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.552875 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.554167 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.554954 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.556507 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.556914 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.557055 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.558522 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:24.558527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:24.558530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:24.558550 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:24.558582 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:24.558595 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:24.558691 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:24.558947 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 11:45:24.559048 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:45:24.560433 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.549104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38411 (local address 127.20.93.131:38515)
0810 11:45:24.550281 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:24.550336 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:24.552389 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:24.553116 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:24.553124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:24.553274 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:24.553976 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:24.553994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.555128 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.555132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:24.556357 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:24.556368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.557163 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.557170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:24.557406 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.558169 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:24.558172 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:24.558211 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:24.558297 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:24.558708 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:24.558722 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:24.558734 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 11:45:24.558916 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 11:45:24.559334 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":421,"threads_started":1}
I20260810 11:45:24.561944 31468 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38411
I20260810 11:45:24.562278 31468 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:24.563083 31468 heartbeater.cc:511] Master 127.20.93.190:38411 requested a full tablet report, sending...
I20260810 11:45:24.564632 30980 ts_manager.cc:194] Registered new tserver with Master: 3fe14343fff943b48ad9390ee97e97cd (127.20.93.131:38763)
I20260810 11:45:24.565459 30980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.131:38515
I20260810 11:45:24.565941 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:45:24.572506 31182 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:24.570690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.20.93.190:38411)
0810 11:45:24.570945 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.570954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.571412 (+ 458us) server_negotiation.cc:407] Connection header received
0810 11:45:24.571609 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.571613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.571676 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.571759 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.572274 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51260: BlockingRecv error: recv got EOF from 127.0.0.1:51260 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:45:24.573208 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31045
I20260810 11:45:24.585104 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.574415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.20.93.129:36491)
0810 11:45:24.578380 (+ 3965us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.578387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.578414 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:45:24.578471 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.578476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.578671 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.578858 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.579344 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.580336 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.581633 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.582239 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.582408 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.583650 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:24.583660 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:24.583672 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:24.584244 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:24.584291 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:24.584311 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:24.584451 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:24.584608 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 11:45:24.584745 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3820,"thread_start_us":3703,"threads_started":1}
I20260810 11:45:24.586534 31175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:24.668649 31180 heartbeater.cc:503] Master 127.20.93.190:38411 was elected leader, sending a full tablet report...
I20260810 11:45:24.669965 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31045
W20260810 11:45:24.689846 30963 connection.cc:535] server connection from 127.20.93.129:45053 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 11:45:24.690284 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31184
I20260810 11:45:24.699059 31479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.691322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.20.93.130:46291)
0810 11:45:24.691875 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.691883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.691910 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:45:24.691978 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.691985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.692207 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.692395 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.692868 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.693974 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.695284 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.695885 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.696048 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.697140 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:24.697148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:24.697162 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:24.697999 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:24.698051 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:24.698087 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:24.698217 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:24.698391 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 11:45:24.698535 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":285,"threads_started":1}
I20260810 11:45:24.700587 31318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:24.784950 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31184
I20260810 11:45:24.803596 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31328
I20260810 11:45:24.812187 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:24.804745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.20.93.131:38763)
0810 11:45:24.805464 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:24.805471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:24.805501 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:45:24.805643 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:24.805655 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:24.805923 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:24.806058 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:24.806466 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.807547 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.808810 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:24.809407 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:24.809517 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:24.810717 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:24.810725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:24.810740 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:24.811350 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:24.811397 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:24.811417 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:24.811507 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:24.811655 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 11:45:24.811797 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":402,"threads_started":1}
I20260810 11:45:24.813463 31462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:24.896167 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31328
I20260810 11:45:24.915292 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 30946
I20260810 11:45:24.916654 31013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:25.022125 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 30946
2026-08-10T11:45:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2631 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T11:45:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:25Z Disabled control of system clock
I20260810 11:45:25.112121 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34301
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:34301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 11:45:25.454862 31487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:25.455178 31487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:25.455324 31487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:25.462792 31487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:25.462898 31487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:25.462999 31487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:25.463116 31487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:25.463203 31487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:25.472496 31487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:34301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:25.473910 31487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:25.475751 31487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:25.488121 31492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:25.488152 31495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:25.488202 31493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:25.489197 31487 server_base.cc:1034] running on GCE node
I20260810 11:45:25.490087 31487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:25.491595 31487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:25.492806 31487 hybrid_clock.cc:648] HybridClock initialized: now 1786362325492760 us; error 34 us; skew 500 ppm
I20260810 11:45:25.493168 31487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:25.495662 31487 webserver.cc:460] Webserver started at http://127.20.93.190:44523/ using document root <none> and password file <none>
I20260810 11:45:25.496399 31487 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:25.496490 31487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:25.496805 31487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:25.500072 31487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "72568339a2544187b73ea0dae98a316f"
format_stamp: "Formatted at 2026-08-10 11:45:25 on dist-test-slave-njxd"
server_key: "8d48d81cdad3503f42ed90f133e792d8"
server_key_iv: "aca6d1a03ab1899bf5e1a9fdec6fa106"
server_key_version: "encryptionkey@0"
I20260810 11:45:25.500991 31487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "72568339a2544187b73ea0dae98a316f"
format_stamp: "Formatted at 2026-08-10 11:45:25 on dist-test-slave-njxd"
server_key: "8d48d81cdad3503f42ed90f133e792d8"
server_key_iv: "aca6d1a03ab1899bf5e1a9fdec6fa106"
server_key_version: "encryptionkey@0"
I20260810 11:45:25.506486 31487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 11:45:25.510001 31501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:25.511322 31487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 11:45:25.511516 31487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "72568339a2544187b73ea0dae98a316f"
format_stamp: "Formatted at 2026-08-10 11:45:25 on dist-test-slave-njxd"
server_key: "8d48d81cdad3503f42ed90f133e792d8"
server_key_iv: "aca6d1a03ab1899bf5e1a9fdec6fa106"
server_key_version: "encryptionkey@0"
I20260810 11:45:25.511756 31487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:25.517740 31487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:25.518532 31487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:25.518865 31487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:25.537464 31487 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:34301
I20260810 11:45:25.537494 31559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:34301 every 8 connection(s)
I20260810 11:45:25.538885 31487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:45:25.540002 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31487
I20260810 11:45:25.540253 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:45:25.540899 20854 external_mini_cluster.cc:1428] Setting key a762f236f0f97a1568c7badb19cdb8f2
I20260810 11:45:25.543262 31560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:25.549417 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:25.543007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.20.93.190:34301)
0810 11:45:25.544044 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:25.544061 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:25.544102 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:45:25.545781 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:25.545803 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:25.546224 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:25.546555 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:25.547739 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:25.547745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:25.547758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:25.547772 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:25.547861 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:25.547879 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:25.548080 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:25.548181 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 11:45:25.548474 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":379,"threads_started":1}
I20260810 11:45:25.555563 31560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f: Bootstrap starting.
I20260810 11:45:25.558933 31560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:25.559927 31560 log.cc:824] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:25.562630 31560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f: No bootstrap required, opened a new log
I20260810 11:45:25.569896 31560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } }
I20260810 11:45:25.570255 31560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:25.570375 31560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72568339a2544187b73ea0dae98a316f, State: Initialized, Role: FOLLOWER
I20260810 11:45:25.570983 31560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } }
I20260810 11:45:25.571306 31560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:45:25.571442 31560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:45:25.571578 31560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:25.573761 31560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } }
I20260810 11:45:25.574201 31560 leader_election.cc:302] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72568339a2544187b73ea0dae98a316f; no voters:
I20260810 11:45:25.574736 31560 leader_election.cc:288] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:45:25.574887 31566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:45:25.575604 31566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [term 1 LEADER]: Becoming Leader. State: Replica: 72568339a2544187b73ea0dae98a316f, State: Running, Role: LEADER
I20260810 11:45:25.576118 31566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } }
I20260810 11:45:25.576921 31560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:45:25.579167 31567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72568339a2544187b73ea0dae98a316f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } } }
I20260810 11:45:25.579623 31567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:25.581898 31568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72568339a2544187b73ea0dae98a316f. Latest consensus state: current_term: 1 leader_uuid: "72568339a2544187b73ea0dae98a316f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72568339a2544187b73ea0dae98a316f" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 34301 } } }
I20260810 11:45:25.583581 31568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:25.584108 31571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:45:25.588765 31571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:45:25.597146 31571 catalog_manager.cc:1347] Generated new cluster ID: b76a7fd06020476b859a0583ef36c258
I20260810 11:45:25.597313 31571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:45:25.611582 31571 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:45:25.612007 31571 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:45:25.618772 31571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72568339a2544187b73ea0dae98a316f: Generated new TSK 0
I20260810 11:45:25.619719 31571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:45:25.634613 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--builtin_ntp_servers=127.20.93.148:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 11:45:25.983948 31585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:25.984308 31585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:25.984483 31585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:25.991489 31585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:25.991637 31585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:25.991839 31585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:45:26.001186 31585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:26.002734 31585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:26.004544 31585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:26.018324 31590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.018326 31591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.019212 31593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:26.020066 31585 server_base.cc:1034] running on GCE node
I20260810 11:45:26.020776 31585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:26.021811 31585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:26.023046 31585 hybrid_clock.cc:648] HybridClock initialized: now 1786362326022999 us; error 42 us; skew 500 ppm
I20260810 11:45:26.023389 31585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:26.031375 31585 webserver.cc:460] Webserver started at http://127.20.93.129:38681/ using document root <none> and password file <none>
I20260810 11:45:26.032159 31585 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:26.032300 31585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:26.032653 31585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:26.035734 31585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "858cd38b24cd4bc6999e0d55c72c9a23"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "61014624c3d78a5c4966f60c03e6e579"
server_key_iv: "b49ea8bf6cbe44726ba88b390744fd52"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.036561 31585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "858cd38b24cd4bc6999e0d55c72c9a23"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "61014624c3d78a5c4966f60c03e6e579"
server_key_iv: "b49ea8bf6cbe44726ba88b390744fd52"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.042217 31585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 11:45:26.045789 31599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.047158 31585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:45:26.047364 31585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "858cd38b24cd4bc6999e0d55c72c9a23"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "61014624c3d78a5c4966f60c03e6e579"
server_key_iv: "b49ea8bf6cbe44726ba88b390744fd52"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.047642 31585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:26.053864 31585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:26.054744 31585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:26.055048 31585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:26.055998 31585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:26.057475 31585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:26.057636 31585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.057770 31585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:26.057862 31585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.092343 31585 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:36445
I20260810 11:45:26.092402 31718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:36445 every 8 connection(s)
I20260810 11:45:26.093876 31585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:45:26.095498 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31585
I20260810 11:45:26.095734 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:45:26.096417 20854 external_mini_cluster.cc:1428] Setting key 4b2b6c0ee9fda076634cdc2629cccf53
I20260810 11:45:26.099488 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--builtin_ntp_servers=127.20.93.148:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 11:45:26.104231 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:26.096606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:57121 (local address 127.20.93.190:34301)
0810 11:45:26.097159 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:26.097163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:26.101656 (+ 4493us) server_negotiation.cc:407] Connection header received
0810 11:45:26.101735 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:26.101743 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:26.101814 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:26.101939 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:26.103297 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:26.103306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:26.103313 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:26.103336 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:26.103393 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:26.103413 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:26.103566 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:26.103758 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 11:45:26.103835 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":257,"threads_started":1}
I20260810 11:45:26.104923 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:26.097362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34301 (local address 127.20.93.129:57121)
0810 11:45:26.098506 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:26.098552 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:26.100191 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:26.102152 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:26.102160 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:26.102258 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:26.102979 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:26.102982 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:26.103020 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:26.103109 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:26.103517 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:26.103525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:26.103530 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 11:45:26.103749 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 11:45:26.103950 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":364,"threads_started":1}
I20260810 11:45:26.106508 31720 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34301
I20260810 11:45:26.106813 31720 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:26.107414 31720 heartbeater.cc:511] Master 127.20.93.190:34301 requested a full tablet report, sending...
I20260810 11:45:26.109488 31520 ts_manager.cc:194] Registered new tserver with Master: 858cd38b24cd4bc6999e0d55c72c9a23 (127.20.93.129:36445)
W20260810 11:45:26.443987 31724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:26.444280 31724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:26.444440 31724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:26.452040 31724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:26.452180 31724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:26.452378 31724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:45:26.461735 31724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:26.463083 31724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:26.464923 31724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:26.478629 31729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.478565 31730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.479130 31732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:26.480043 31724 server_base.cc:1034] running on GCE node
I20260810 11:45:26.480680 31724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:26.481729 31724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:26.482956 31724 hybrid_clock.cc:648] HybridClock initialized: now 1786362326482906 us; error 44 us; skew 500 ppm
I20260810 11:45:26.483299 31724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:26.485832 31724 webserver.cc:460] Webserver started at http://127.20.93.130:37727/ using document root <none> and password file <none>
I20260810 11:45:26.486562 31724 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:26.486683 31724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:26.487049 31724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:26.490115 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "03a02ab870ed4c25b5308805811d11d4"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "119a590f28817fb2d71e3fb57853c40c"
server_key_iv: "0a32c133e416ab072fb7d3dc9f37e775"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.490938 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "03a02ab870ed4c25b5308805811d11d4"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "119a590f28817fb2d71e3fb57853c40c"
server_key_iv: "0a32c133e416ab072fb7d3dc9f37e775"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.496181 31724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:45:26.499787 31738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.501204 31724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:45:26.501368 31724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "03a02ab870ed4c25b5308805811d11d4"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "119a590f28817fb2d71e3fb57853c40c"
server_key_iv: "0a32c133e416ab072fb7d3dc9f37e775"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.501621 31724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:26.507462 31724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:26.508280 31724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:26.508576 31724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:26.509560 31724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:26.511050 31724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:26.511170 31724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.511307 31724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:26.511401 31724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.545786 31724 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:36787
I20260810 11:45:26.545846 31858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:36787 every 8 connection(s)
I20260810 11:45:26.547220 31724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:45:26.550513 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31724
I20260810 11:45:26.550720 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:45:26.551334 20854 external_mini_cluster.cc:1428] Setting key 3bb0732502ab5598fd34159f5279ee26
I20260810 11:45:26.554528 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--builtin_ntp_servers=127.20.93.148:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 11:45:26.557443 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:26.550267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:38953 (local address 127.20.93.190:34301)
0810 11:45:26.550479 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:26.550488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:26.552385 (+ 1897us) server_negotiation.cc:407] Connection header received
0810 11:45:26.555073 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:26.555079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:26.555170 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:26.555289 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:26.556661 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:26.556666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:26.556671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:26.556689 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:26.556748 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:26.556760 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:26.556886 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:26.557087 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 11:45:26.557160 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:45:26.558473 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:26.550970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34301 (local address 127.20.93.130:38953)
0810 11:45:26.552212 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:26.552259 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:26.554779 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:26.555538 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:26.555546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:26.555641 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:26.556337 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:26.556339 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:26.556379 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:26.556481 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:26.556859 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:26.556870 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:26.556878 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:45:26.557186 (+ 308us) client_negotiation.cc:240] Negotiation successful
0810 11:45:26.557459 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":401,"threads_started":1}
I20260810 11:45:26.559859 31859 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34301
I20260810 11:45:26.560205 31859 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:26.560930 31859 heartbeater.cc:511] Master 127.20.93.190:34301 requested a full tablet report, sending...
I20260810 11:45:26.562633 31520 ts_manager.cc:194] Registered new tserver with Master: 03a02ab870ed4c25b5308805811d11d4 (127.20.93.130:36787)
W20260810 11:45:26.898147 31862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:26.898502 31862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:26.898675 31862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:26.906243 31862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:26.906373 31862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:26.906543 31862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:45:26.915746 31862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:26.917157 31862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:26.919003 31862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:26.932523 31867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.932665 31871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:26.932588 31868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:26.933606 31862 server_base.cc:1034] running on GCE node
I20260810 11:45:26.934226 31862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:26.935252 31862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:26.936482 31862 hybrid_clock.cc:648] HybridClock initialized: now 1786362326936430 us; error 47 us; skew 500 ppm
I20260810 11:45:26.936816 31862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:45:26.939101 31862 webserver.cc:460] Webserver started at http://127.20.93.131:46479/ using document root <none> and password file <none>
I20260810 11:45:26.939821 31862 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:26.939957 31862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:26.940294 31862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:26.943224 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "0465a5437d464a64ae27007af4e329b1"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "175be6ac6ab71a92a035b630ffd21954"
server_key_iv: "66a5c587cfcd632102eefb167914302c"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.944031 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "0465a5437d464a64ae27007af4e329b1"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "175be6ac6ab71a92a035b630ffd21954"
server_key_iv: "66a5c587cfcd632102eefb167914302c"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.949393 31862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:45:26.953167 31877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.954543 31862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:45:26.954720 31862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "0465a5437d464a64ae27007af4e329b1"
format_stamp: "Formatted at 2026-08-10 11:45:26 on dist-test-slave-njxd"
server_key: "175be6ac6ab71a92a035b630ffd21954"
server_key_iv: "66a5c587cfcd632102eefb167914302c"
server_key_version: "encryptionkey@0"
I20260810 11:45:26.954938 31862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:26.961023 31862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:26.961954 31862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:26.962239 31862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:26.963217 31862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:26.964655 31862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:26.964771 31862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.964876 31862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:26.964951 31862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:26.998505 31862 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:44307
I20260810 11:45:26.998556 31992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:44307 every 8 connection(s)
I20260810 11:45:26.999936 31862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:45:27.006006 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 31862
I20260810 11:45:27.006240 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:45:27.006703 20854 external_mini_cluster.cc:1428] Setting key 3d71cc86409d30b88a1f9c1ad5f8337e
I20260810 11:45:27.008952 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:27.002732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:41581 (local address 127.20.93.190:34301)
0810 11:45:27.002921 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:27.002926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:27.004428 (+ 1502us) server_negotiation.cc:407] Connection header received
0810 11:45:27.006387 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:27.006394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:27.006486 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:27.006662 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:27.008141 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:27.008149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:27.008158 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:27.008176 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:27.008212 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:27.008225 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:27.008346 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:27.008560 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 11:45:27.008643 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:45:27.010068 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:27.003299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:34301 (local address 127.20.93.131:41581)
0810 11:45:27.004247 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:27.004298 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:27.006045 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:27.006915 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:27.006925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:27.007029 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:27.007822 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:45:27.007825 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:27.007867 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:45:27.007963 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:27.008335 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:27.008347 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:27.008357 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:45:27.008650 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:45:27.008936 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":291,"threads_started":1}
I20260810 11:45:27.011282 31993 heartbeater.cc:347] Connected to a master server at 127.20.93.190:34301
I20260810 11:45:27.011552 31993 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:27.012094 31993 heartbeater.cc:511] Master 127.20.93.190:34301 requested a full tablet report, sending...
I20260810 11:45:27.013393 31520 ts_manager.cc:194] Registered new tserver with Master: 0465a5437d464a64ae27007af4e329b1 (127.20.93.131:44307)
I20260810 11:45:27.022406 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:45:27.028836 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:45:27.027330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.20.93.190:34301)
0810 11:45:27.027497 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:27.027502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:27.028053 (+ 551us) server_negotiation.cc:407] Connection header received
0810 11:45:27.028204 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:27.028208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:27.028261 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:27.028345 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:27.028630 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58098: BlockingRecv error: recv got EOF from 127.0.0.1:58098 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:45:27.029965 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31585
I20260810 11:45:27.033806 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:27.030941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.20.93.129:36445)
0810 11:45:27.031481 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:27.031488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:27.031513 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:45:27.031571 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:27.031577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:27.031793 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:27.031964 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:27.033006 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:27.033012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:27.033025 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:27.033051 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:27.033091 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:27.033105 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:27.033250 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:27.033357 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 11:45:27.033461 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":234,"threads_started":1}
I20260810 11:45:27.035051 31714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:27.122531 31720 heartbeater.cc:503] Master 127.20.93.190:34301 was elected leader, sending a full tablet report...
I20260810 11:45:27.125798 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31585
I20260810 11:45:27.146636 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31724
I20260810 11:45:27.150395 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:27.147620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.20.93.130:36787)
0810 11:45:27.148129 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:27.148147 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:27.148173 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:45:27.148234 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:27.148241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:27.148460 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:27.148586 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:27.149562 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:27.149569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:27.149631 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:27.149659 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:27.149704 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:27.149720 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:27.149829 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:27.149979 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 11:45:27.150103 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":240,"threads_started":1}
I20260810 11:45:27.151679 31854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:27.234994 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31724
I20260810 11:45:27.254190 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31862
I20260810 11:45:27.258579 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:27.255273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.20.93.131:44307)
0810 11:45:27.255907 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:27.255914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:27.255945 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:45:27.256010 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:27.256017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:27.256248 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:27.256437 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:27.257725 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:27.257732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:27.257744 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:45:27.257763 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:27.257806 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:45:27.257820 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:27.258014 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:27.258093 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 11:45:27.258213 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":320,"threads_started":1}
I20260810 11:45:27.259897 31988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:27.344148 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31862
I20260810 11:45:27.362949 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 31487
I20260810 11:45:27.364274 31555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:27.469046 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 31487
2026-08-10T11:45:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2446 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:45:27 dist-test-slave-njxd krb5kdc[32013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:45:27 dist-test-slave-njxd krb5kdc[32013](info): set up 2 sockets
Aug 10 11:45:27 dist-test-slave-njxd krb5kdc[32013](info): commencing operation
krb5kdc: starting...
W20260810 11:45:28.645845 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:45:28 dist-test-slave-njxd krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:45:28.949700 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43623
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:29.309871 32028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:29.310216 32028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:29.310310 32028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:29.317953 32028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:45:29.318089 32028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:29.318181 32028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:29.318295 32028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:45:29.318382 32028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:45:29.328927 32028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:29.330528 32028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:29.332229 32028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:29.346580 32034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:29.346657 32037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:29.346607 32035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:29.347283 32028 server_base.cc:1034] running on GCE node
I20260810 11:45:29.348276 32028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:29.350251 32028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:29.351576 32028 hybrid_clock.cc:648] HybridClock initialized: now 1786362329351496 us; error 51 us; skew 500 ppm
I20260810 11:45:29.352034 32028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:29 dist-test-slave-njxd krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362329, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:29.355655 32028 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:45:29.357467 32028 webserver.cc:460] Webserver started at http://127.20.93.190:41665/ using document root <none> and password file <none>
I20260810 11:45:29.358301 32028 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:29.358466 32028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:29.358824 32028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:29.362231 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "08ab2f6928b9415f9e4adfe879476ebb"
format_stamp: "Formatted at 2026-08-10 11:45:29 on dist-test-slave-njxd"
server_key: "297f118e5131f790653691f8d34eae72"
server_key_iv: "dd6890ea0637dd659b769317db562fa1"
server_key_version: "encryptionkey@0"
I20260810 11:45:29.363078 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "08ab2f6928b9415f9e4adfe879476ebb"
format_stamp: "Formatted at 2026-08-10 11:45:29 on dist-test-slave-njxd"
server_key: "297f118e5131f790653691f8d34eae72"
server_key_iv: "dd6890ea0637dd659b769317db562fa1"
server_key_version: "encryptionkey@0"
I20260810 11:45:29.368752 32028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:45:29.372787 32045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:29.374598 32028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:45:29.374811 32028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "08ab2f6928b9415f9e4adfe879476ebb"
format_stamp: "Formatted at 2026-08-10 11:45:29 on dist-test-slave-njxd"
server_key: "297f118e5131f790653691f8d34eae72"
server_key_iv: "dd6890ea0637dd659b769317db562fa1"
server_key_version: "encryptionkey@0"
I20260810 11:45:29.375033 32028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:29.412010 32028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:29.416801 32028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:29.417166 32028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:29.437371 32028 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:43623
I20260810 11:45:29.437407 32097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:43623 every 8 connection(s)
I20260810 11:45:29.439021 32028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:45:29.441162 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 32028
I20260810 11:45:29.441452 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:45:29.442411 20854 external_mini_cluster.cc:1428] Setting key 03553ba47b1bddba4f1cbbd2f9648458
I20260810 11:45:29.444402 32098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:45:29 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:29.459982 32098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: Bootstrap starting.
I20260810 11:45:29.463601 32098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:29.465045 32098 log.cc:824] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:29.468720 32098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: No bootstrap required, opened a new log
I20260810 11:45:29.469029 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:29.444871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.20.93.190:43623)
0810 11:45:29.445827 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:29.445840 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:29.445885 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:45:29.447579 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:29.447603 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:29.448084 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:29.448498 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:29.450717 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:29.452311 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:29.454159 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:29.454909 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:29.455070 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:29.460455 (+ 5385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:29.460465 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:29.460481 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:29.460525 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:29.463958 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:29.464943 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:29.464950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:29.464957 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:29.465138 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:29.465758 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:29.465766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:29.465772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:29.466676 (+ 904us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:29.466952 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:29.467454 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 11:45:29.467919 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":416,"threads_started":1}
I20260810 11:45:29.477564 32098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } }
I20260810 11:45:29.478104 32098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:29.478226 32098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ab2f6928b9415f9e4adfe879476ebb, State: Initialized, Role: FOLLOWER
I20260810 11:45:29.478952 32098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } }
I20260810 11:45:29.479247 32098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:45:29.479422 32098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:45:29.479640 32098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:29.482241 32098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } }
I20260810 11:45:29.482812 32098 leader_election.cc:302] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08ab2f6928b9415f9e4adfe879476ebb; no voters:
I20260810 11:45:29.483562 32098 leader_election.cc:288] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:45:29.483708 32103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:45:29.484601 32103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [term 1 LEADER]: Becoming Leader. State: Replica: 08ab2f6928b9415f9e4adfe879476ebb, State: Running, Role: LEADER
I20260810 11:45:29.485219 32103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } }
I20260810 11:45:29.486186 32098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:45:29.488158 32105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08ab2f6928b9415f9e4adfe879476ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } } }
I20260810 11:45:29.488678 32105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:29.489332 32104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08ab2f6928b9415f9e4adfe879476ebb. Latest consensus state: current_term: 1 leader_uuid: "08ab2f6928b9415f9e4adfe879476ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ab2f6928b9415f9e4adfe879476ebb" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43623 } } }
I20260810 11:45:29.489691 32104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb [sys.catalog]: This master's current role is: LEADER
I20260810 11:45:29.490648 32108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:45:29.496241 32108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:45:29.506446 32108 catalog_manager.cc:1347] Generated new cluster ID: c7e3e80b002641f1aaab1382ffb1406e
I20260810 11:45:29.506711 32108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:45:29.528506 32108 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:45:29.530030 32108 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:45:29.539564 32108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: Generated new TSK 0
I20260810 11:45:29.540637 32108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:45:29.617774 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:29.975026 32125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:29.975330 32125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:29.975515 32125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:29.983223 32125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:29.983349 32125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:29.983562 32125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:45:29.994259 32125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:29.995697 32125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:29.997078 32125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:30.010977 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:30.011085 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:30.011073 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:30.011932 32125 server_base.cc:1034] running on GCE node
I20260810 11:45:30.012598 32125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:30.013656 32125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:30.014914 32125 hybrid_clock.cc:648] HybridClock initialized: now 1786362330014841 us; error 64 us; skew 500 ppm
I20260810 11:45:30.015295 32125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:30 dist-test-slave-njxd krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362330, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:30.018788 32125 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:45:30.020447 32125 webserver.cc:460] Webserver started at http://127.20.93.129:39003/ using document root <none> and password file <none>
I20260810 11:45:30.021193 32125 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:30.021347 32125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:30.021749 32125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:30.024868 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "f2c6cc33ae694e11b447270929b3a83b"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "a7a72ef7e107cd202115ee3d5c38b7ea"
server_key_iv: "09ccf1dabe3651ebd6f010f43c2af79e"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.025745 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "f2c6cc33ae694e11b447270929b3a83b"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "a7a72ef7e107cd202115ee3d5c38b7ea"
server_key_iv: "09ccf1dabe3651ebd6f010f43c2af79e"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.031195 32125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:45:30.035087 32141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:30.036808 32125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:45:30.037017 32125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "f2c6cc33ae694e11b447270929b3a83b"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "a7a72ef7e107cd202115ee3d5c38b7ea"
server_key_iv: "09ccf1dabe3651ebd6f010f43c2af79e"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.037257 32125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:30.058593 32125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:30.062736 32125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:30.063061 32125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:30.064076 32125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:30.065536 32125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:30.065675 32125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:45:30.065783 32125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:30.065830 32125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:30.101585 32125 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:46649
I20260810 11:45:30.101665 32261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:46649 every 8 connection(s)
I20260810 11:45:30.102982 32125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:45:30.108978 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 32125
I20260810 11:45:30.109210 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:45:30.109953 20854 external_mini_cluster.cc:1428] Setting key 8d8d04ddcb2de70a0b3fc41776129dc0
Aug 10 11:45:30 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362330, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:30.126720 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:30.105777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:32931 (local address 127.20.93.190:43623)
0810 11:45:30.106319 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:30.106323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:30.107424 (+ 1101us) server_negotiation.cc:407] Connection header received
0810 11:45:30.109459 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:30.109465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:30.109563 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:30.109712 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:30.111550 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.112465 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.114211 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.114626 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.114772 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:30.119366 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:30.119375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:30.119382 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:30.119427 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:30.122919 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:30.123854 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:30.123862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:30.123867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:30.124040 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:30.124481 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:30.124489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:30.124492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:30.124903 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:30.125068 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:30.125905 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 11:45:30.126197 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":277,"threads_started":1}
I20260810 11:45:30.127652 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:30.106432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43623 (local address 127.20.93.129:32931)
0810 11:45:30.107237 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:30.107278 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:30.109092 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:30.110014 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:30.110029 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:30.110717 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:30.111368 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:30.111387 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.112643 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.112648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:30.113996 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:30.114009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.114911 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.114920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:30.115238 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:30.116059 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:30.116098 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:30.119077 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:30.123096 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:30.123107 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:30.123127 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:30.123709 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:30.124161 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:30.124165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:30.124167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:30.124351 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:30.125051 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:30.125058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:30.125393 (+ 335us) client_negotiation.cc:769] Sending connection context
0810 11:45:30.125738 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 11:45:30.126182 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":311,"threads_started":1}
I20260810 11:45:30.129408 32262 heartbeater.cc:347] Connected to a master server at 127.20.93.190:43623
I20260810 11:45:30.129791 32262 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:30.130786 32262 heartbeater.cc:511] Master 127.20.93.190:43623 requested a full tablet report, sending...
I20260810 11:45:30.133157 32062 ts_manager.cc:194] Registered new tserver with Master: f2c6cc33ae694e11b447270929b3a83b (127.20.93.129:46649)
I20260810 11:45:30.134758 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:32931
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:45:30.178887 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:30.539438 32268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:30.539914 32268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:30.540064 32268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:30.548655 32268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:30.548775 32268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:30.548893 32268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:45:30.559134 32268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:30.560442 32268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:30.561873 32268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:30.575420 32273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:30.575383 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:30.575472 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:30.576738 32268 server_base.cc:1034] running on GCE node
I20260810 11:45:30.577517 32268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:30.578662 32268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:30.579867 32268 hybrid_clock.cc:648] HybridClock initialized: now 1786362330579808 us; error 53 us; skew 500 ppm
I20260810 11:45:30.580262 32268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:30 dist-test-slave-njxd krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362330, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:30.583583 32268 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:45:30.585111 32268 webserver.cc:460] Webserver started at http://127.20.93.130:46203/ using document root <none> and password file <none>
I20260810 11:45:30.586091 32268 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:30.586256 32268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:30.586588 32268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:30.590032 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "14289f14acfb44e196d145e10e699131"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "aa759e3c0f7d3b108109b71a167adee3"
server_key_iv: "c6237e85348f44b84ce885ff9c1f7364"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.590878 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "14289f14acfb44e196d145e10e699131"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "aa759e3c0f7d3b108109b71a167adee3"
server_key_iv: "c6237e85348f44b84ce885ff9c1f7364"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.596804 32268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:45:30.600852 32285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:30.602455 32268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:45:30.602676 32268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "14289f14acfb44e196d145e10e699131"
format_stamp: "Formatted at 2026-08-10 11:45:30 on dist-test-slave-njxd"
server_key: "aa759e3c0f7d3b108109b71a167adee3"
server_key_iv: "c6237e85348f44b84ce885ff9c1f7364"
server_key_version: "encryptionkey@0"
I20260810 11:45:30.602907 32268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:30.634505 32268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:30.639127 32268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:30.639475 32268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:30.640492 32268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:30.642076 32268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:30.642153 32268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:30.642241 32268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:30.642293 32268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:30.677997 32268 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:37671
I20260810 11:45:30.678085 32404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:37671 every 8 connection(s)
I20260810 11:45:30.679572 32268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:45:30.680310 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 32268
I20260810 11:45:30.680552 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:45:30.681169 20854 external_mini_cluster.cc:1428] Setting key 805fb4162557113aab239d303c50f4c9
Aug 10 11:45:30 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362330, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:30.701743 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:30.682758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:50431 (local address 127.20.93.190:43623)
0810 11:45:30.683255 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:30.683261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:30.684482 (+ 1221us) server_negotiation.cc:407] Connection header received
0810 11:45:30.686812 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:30.686819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:30.686909 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:30.687055 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:30.688731 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.689472 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.691123 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.691567 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.691708 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:30.695712 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:30.695717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:30.695720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:30.695754 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:30.698486 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:30.699262 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:30.699268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:30.699270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:30.699401 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:30.699868 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:30.699872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:30.699874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:30.700149 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:30.700323 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:30.701072 (+ 749us) server_negotiation.cc:299] Negotiation successful
0810 11:45:30.701320 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260810 11:45:30.702775 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:30.683502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43623 (local address 127.20.93.130:50431)
0810 11:45:30.684318 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:30.684366 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:30.686511 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:30.687328 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:30.687338 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:30.687941 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:30.688541 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:30.688559 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.689676 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.689680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:30.690948 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:30.690960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:30.691850 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:30.691862 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:30.692096 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:30.692878 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:30.692905 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:30.695460 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:30.698647 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:30.698657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:30.698674 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:30.699122 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:30.699519 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:30.699524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:30.699528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:30.699736 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:30.700292 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:30.700300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:30.700688 (+ 388us) client_negotiation.cc:769] Sending connection context
0810 11:45:30.700961 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 11:45:30.701565 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":341,"threads_started":1}
I20260810 11:45:30.704345 32405 heartbeater.cc:347] Connected to a master server at 127.20.93.190:43623
I20260810 11:45:30.704698 32405 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:30.705471 32405 heartbeater.cc:511] Master 127.20.93.190:43623 requested a full tablet report, sending...
I20260810 11:45:30.707062 32062 ts_manager.cc:194] Registered new tserver with Master: 14289f14acfb44e196d145e10e699131 (127.20.93.130:37671)
I20260810 11:45:30.708037 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:50431
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:45:30.749531 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:45:31.098374 32411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:45:31.098687 32411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:45:31.098841 32411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:45:31.108880 32411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:45:31.108990 32411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:45:31.109176 32411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:45:31.119225 32411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:45:31.120561 32411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:45:31.121955 32411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:45:31.135187 32419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:31.135308 32416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:45:31.135281 32417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:45:31.136291 32411 server_base.cc:1034] running on GCE node
I20260810 11:45:31.136991 32411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:45:31.138109 32411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:45:31.138729 32262 heartbeater.cc:503] Master 127.20.93.190:43623 was elected leader, sending a full tablet report...
I20260810 11:45:31.139353 32411 hybrid_clock.cc:648] HybridClock initialized: now 1786362331139284 us; error 56 us; skew 500 ppm
I20260810 11:45:31.139756 32411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:45:31 dist-test-slave-njxd krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362331, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:45:31.147768 32411 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:45:31.149480 32411 webserver.cc:460] Webserver started at http://127.20.93.131:37679/ using document root <none> and password file <none>
I20260810 11:45:31.150247 32411 fs_manager.cc:359] Metadata directory not provided
I20260810 11:45:31.150396 32411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:45:31.150720 32411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:45:31.153815 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "7548a96511494929aaebabb4fc679c61"
format_stamp: "Formatted at 2026-08-10 11:45:31 on dist-test-slave-njxd"
server_key: "44d869bfb994a62496f91322b7a63785"
server_key_iv: "104fdc1482b8f55620b5f0173ba552a8"
server_key_version: "encryptionkey@0"
I20260810 11:45:31.154657 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "7548a96511494929aaebabb4fc679c61"
format_stamp: "Formatted at 2026-08-10 11:45:31 on dist-test-slave-njxd"
server_key: "44d869bfb994a62496f91322b7a63785"
server_key_iv: "104fdc1482b8f55620b5f0173ba552a8"
server_key_version: "encryptionkey@0"
I20260810 11:45:31.160102 32411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 11:45:31.163699 32426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:31.165128 32411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:45:31.165318 32411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "7548a96511494929aaebabb4fc679c61"
format_stamp: "Formatted at 2026-08-10 11:45:31 on dist-test-slave-njxd"
server_key: "44d869bfb994a62496f91322b7a63785"
server_key_iv: "104fdc1482b8f55620b5f0173ba552a8"
server_key_version: "encryptionkey@0"
I20260810 11:45:31.165526 32411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:45:31.185148 32411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:45:31.190665 32411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:45:31.191031 32411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:45:31.192153 32411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:45:31.193677 32411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:45:31.193755 32411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:31.193907 32411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:45:31.193957 32411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:45:31.228241 32411 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:35823
I20260810 11:45:31.228286 32545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:35823 every 8 connection(s)
I20260810 11:45:31.229668 32411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:45:31.230029 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 32411
I20260810 11:45:31.230262 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:45:31.230929 20854 external_mini_cluster.cc:1428] Setting key 6ef2439593be8c0ebcd339089d8c1daf
Aug 10 11:45:31 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362331, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:45:31.250989 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.232677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:50225 (local address 127.20.93.190:43623)
0810 11:45:31.233231 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.233237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.234314 (+ 1077us) server_negotiation.cc:407] Connection header received
0810 11:45:31.236333 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.236338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.236442 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.236571 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:31.238190 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.238993 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.240577 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.240982 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.241125 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.244773 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:31.244778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:31.244781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:31.244826 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:31.248135 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.248837 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.248841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.248843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.248964 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.249354 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.249357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.249359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.249702 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:31.249832 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.250535 (+ 703us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.250716 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":252,"threads_started":1}
I20260810 11:45:31.252005 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.233349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43623 (local address 127.20.93.131:50225)
0810 11:45:31.234162 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.234199 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.236059 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.236830 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.236838 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.237368 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:45:31.238020 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.238038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.239176 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.239180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.240424 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.240436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.241224 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.241233 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.241555 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.242253 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:45:31.242281 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:45:31.244558 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:45:31.248261 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:31.248267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:31.248281 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:31.248719 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:31.249057 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:45:31.249060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:45:31.249063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:45:31.249253 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:45:31.249807 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:45:31.249813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:45:31.250144 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.250381 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.250789 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":360,"threads_started":1}
I20260810 11:45:31.253513 32546 heartbeater.cc:347] Connected to a master server at 127.20.93.190:43623
I20260810 11:45:31.253875 32546 heartbeater.cc:464] Registering TS with master...
I20260810 11:45:31.254658 32546 heartbeater.cc:511] Master 127.20.93.190:43623 requested a full tablet report, sending...
I20260810 11:45:31.256129 32062 ts_manager.cc:194] Registered new tserver with Master: 7548a96511494929aaebabb4fc679c61 (127.20.93.131:35823)
I20260810 11:45:31.256951 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:50225
I20260810 11:45:31.259910 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:45:31.277774 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.264693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.20.93.190:43623)
0810 11:45:31.264864 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.264868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.265342 (+ 474us) server_negotiation.cc:407] Connection header received
0810 11:45:31.265489 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.265493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.265644 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.265737 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:31.267161 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.267935 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.269377 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.269919 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.270027 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.272456 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:31.272461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:31.272465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:31.272500 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:31.275122 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.275830 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.275834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.275837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.275982 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.276336 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.276340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.276342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.276675 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:31.276857 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.277225 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.277410 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:45:31.282263 32062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:45:31.289719 32062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:45:31.316728 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.302760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.20.93.129:46649)
0810 11:45:31.303600 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.303610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.304160 (+ 550us) server_negotiation.cc:407] Connection header received
0810 11:45:31.304511 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.304520 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.304801 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.305007 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:45:31.306459 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.308904 (+ 2445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.312634 (+ 3730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.314767 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.314913 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.315213 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.316111 (+ 898us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.316352 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":356,"threads_started":1}
I20260810 11:45:31.317986 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.302428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46649 (local address 127.0.0.1:48794)
0810 11:45:31.304002 (+ 1574us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.304043 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.304268 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.305472 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.305484 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.305546 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:45:31.306179 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.306202 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.309150 (+ 2948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.309158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.312369 (+ 3211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.312391 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.315437 (+ 3046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.315446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.315761 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.315833 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.317287 (+ 1454us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.317515 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"thread_start_us":1140,"threads_started":1}
I20260810 11:45:31.323616 32195 tablet_service.cc:1467] Processing CreateTablet for tablet 12cca883a699477c8b927c48351a05f9 (DEFAULT_TABLE table=test-table [id=b589ee50699349fe8509a9005aba240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:45:31.325202 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.303500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.20.93.131:35823)
0810 11:45:31.316828 (+ 13328us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.316839 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.316882 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:45:31.316981 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.316989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.317292 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.318438 (+ 1146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:45:31.318746 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.320367 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.322991 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.324141 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.324289 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.324532 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.324715 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.324884 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13121,"thread_start_us":13945,"threads_started":1}
I20260810 11:45:31.325219 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.302881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:35823 (local address 127.0.0.1:36470)
0810 11:45:31.304022 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.304044 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.304234 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.317764 (+ 13530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.317769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.317797 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:45:31.318225 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.318239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.320547 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.320552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.322810 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.322826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.324336 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.324342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.324519 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.324551 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.324669 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.324803 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"mutex_wait_us":25,"thread_start_us":943,"threads_started":1}
I20260810 11:45:31.325445 32195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12cca883a699477c8b927c48351a05f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:31.326108 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.304476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:37671 (local address 127.0.0.1:48776)
0810 11:45:31.318403 (+ 13927us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.318420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.318509 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.319707 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.319712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.319737 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:45:31.319950 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.319959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.321990 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.321998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.323453 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.323468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.325295 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.325300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.325505 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.325538 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.325630 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.325779 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13829,"mutex_wait_us":3,"thread_start_us":14793,"threads_started":1}
I20260810 11:45:31.326653 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.304867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.20.93.130:37671)
0810 11:45:31.319096 (+ 14229us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.319105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.319134 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:45:31.319246 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.319254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.319477 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.320121 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:45:31.320325 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.321781 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.323648 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.325087 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.325854 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.326042 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.326117 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.326263 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14038,"thread_start_us":13880,"threads_started":1}
I20260810 11:45:31.329262 32477 tablet_service.cc:1467] Processing CreateTablet for tablet 12cca883a699477c8b927c48351a05f9 (DEFAULT_TABLE table=test-table [id=b589ee50699349fe8509a9005aba240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:45:31.330683 32477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12cca883a699477c8b927c48351a05f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:31.330284 32334 tablet_service.cc:1467] Processing CreateTablet for tablet 12cca883a699477c8b927c48351a05f9 (DEFAULT_TABLE table=test-table [id=b589ee50699349fe8509a9005aba240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:45:31.331614 32334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12cca883a699477c8b927c48351a05f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:45:31.342097 32565 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Bootstrap starting.
I20260810 11:45:31.346199 32565 tablet_bootstrap.cc:654] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:31.347261 32565 log.cc:824] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:31.348165 32567 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Bootstrap starting.
I20260810 11:45:31.348198 32566 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Bootstrap starting.
I20260810 11:45:31.351667 32566 tablet_bootstrap.cc:654] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:31.352187 32567 tablet_bootstrap.cc:654] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Neither blocks nor log segments found. Creating new log.
I20260810 11:45:31.352734 32566 log.cc:824] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:31.353205 32565 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: No bootstrap required, opened a new log
I20260810 11:45:31.353454 32567 log.cc:824] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Log is configured to *not* fsync() on all Append() calls
I20260810 11:45:31.353602 32565 ts_tablet_manager.cc:1397] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260810 11:45:31.355831 32566 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: No bootstrap required, opened a new log
I20260810 11:45:31.356089 32566 ts_tablet_manager.cc:1397] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 11:45:31.362687 32567 tablet_bootstrap.cc:492] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: No bootstrap required, opened a new log
I20260810 11:45:31.362972 32567 ts_tablet_manager.cc:1397] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260810 11:45:31.363878 32566 raft_consensus.cc:348] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.364248 32566 raft_consensus.cc:374] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:31.364346 32566 raft_consensus.cc:729] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c6cc33ae694e11b447270929b3a83b, State: Initialized, Role: FOLLOWER
I20260810 11:45:31.364985 32566 consensus_queue.cc:260] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.365751 32565 raft_consensus.cc:348] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.366173 32565 raft_consensus.cc:374] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:31.366285 32565 raft_consensus.cc:729] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7548a96511494929aaebabb4fc679c61, State: Initialized, Role: FOLLOWER
I20260810 11:45:31.366871 32565 consensus_queue.cc:260] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.371031 32546 heartbeater.cc:503] Master 127.20.93.190:43623 was elected leader, sending a full tablet report...
I20260810 11:45:31.372095 32565 ts_tablet_manager.cc:1428] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Time spent starting tablet: real 0.018s user 0.003s sys 0.013s
I20260810 11:45:31.373742 32566 ts_tablet_manager.cc:1428] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Time spent starting tablet: real 0.018s user 0.009s sys 0.010s
I20260810 11:45:31.375883 32567 raft_consensus.cc:348] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.376348 32567 raft_consensus.cc:374] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:45:31.376487 32567 raft_consensus.cc:729] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14289f14acfb44e196d145e10e699131, State: Initialized, Role: FOLLOWER
I20260810 11:45:31.377146 32567 consensus_queue.cc:260] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.378373 32405 heartbeater.cc:503] Master 127.20.93.190:43623 was elected leader, sending a full tablet report...
I20260810 11:45:31.379027 32567 ts_tablet_manager.cc:1428] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
W20260810 11:45:31.432138 32406 tablet.cc:2369] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:45:31.446626 32571 raft_consensus.cc:482] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:45:31.446944 32571 raft_consensus.cc:504] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.448752 32571 leader_election.cc:288] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14289f14acfb44e196d145e10e699131 (127.20.93.130:37671), 7548a96511494929aaebabb4fc679c61 (127.20.93.131:35823)
I20260810 11:45:31.458560 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.449329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:44339 (local address 127.20.93.131:35823)
0810 11:45:31.449766 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.449775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.450872 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 11:45:31.451201 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.451208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.451285 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.451388 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:45:31.452050 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.453170 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.456205 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.457488 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.457706 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.457786 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.458132 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.458287 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260810 11:45:31.458632 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.449125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:35823 (local address 127.20.93.129:44339)
0810 11:45:31.450727 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.450753 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.451013 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.451423 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.451430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.451458 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:45:31.451882 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.451893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.453313 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.453317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.456003 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.456019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.457734 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.457740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.457925 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.457963 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.458070 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.458206 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"mutex_wait_us":136,"thread_start_us":1257,"threads_started":1}
I20260810 11:45:31.458763 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.450078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41411 (local address 127.20.93.130:37671)
0810 11:45:31.450358 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.450368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.451270 (+ 902us) server_negotiation.cc:407] Connection header received
0810 11:45:31.451493 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.451500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.451592 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.451662 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:45:31.452408 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.453443 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.456213 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.457486 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.457729 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.457799 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.458347 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.458485 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:45:31.459388 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.449220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:37671 (local address 127.20.93.129:41411)
0810 11:45:31.451151 (+ 1931us) negotiation.cc:107] Waiting for socket to connect
0810 11:45:31.451174 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:45:31.451307 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:45:31.451892 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:45:31.451897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:45:31.451922 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:45:31.452228 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.452237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.453603 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.453611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.456003 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:45:31.456019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.457955 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.457960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:45:31.458153 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.458184 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:45:31.458863 (+ 679us) client_negotiation.cc:240] Negotiation successful
0810 11:45:31.459007 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1798,"mutex_wait_us":72,"thread_start_us":1558,"threads_started":1}
I20260810 11:45:31.460021 32498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12cca883a699477c8b927c48351a05f9" candidate_uuid: "f2c6cc33ae694e11b447270929b3a83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7548a96511494929aaebabb4fc679c61" is_pre_election: true
I20260810 11:45:31.460496 32356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12cca883a699477c8b927c48351a05f9" candidate_uuid: "f2c6cc33ae694e11b447270929b3a83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14289f14acfb44e196d145e10e699131" is_pre_election: true
I20260810 11:45:31.460736 32498 raft_consensus.cc:2454] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6cc33ae694e11b447270929b3a83b in term 0.
I20260810 11:45:31.461079 32356 raft_consensus.cc:2454] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6cc33ae694e11b447270929b3a83b in term 0.
I20260810 11:45:31.461623 32145 leader_election.cc:302] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7548a96511494929aaebabb4fc679c61, f2c6cc33ae694e11b447270929b3a83b; no voters:
I20260810 11:45:31.462117 32571 raft_consensus.cc:2781] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:45:31.462286 32571 raft_consensus.cc:482] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:45:31.462399 32571 raft_consensus.cc:3037] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:31.464671 32571 raft_consensus.cc:504] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.465322 32571 leader_election.cc:288] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [CANDIDATE]: Term 1 election: Requested vote from peers 14289f14acfb44e196d145e10e699131 (127.20.93.130:37671), 7548a96511494929aaebabb4fc679c61 (127.20.93.131:35823)
I20260810 11:45:31.465829 32498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12cca883a699477c8b927c48351a05f9" candidate_uuid: "f2c6cc33ae694e11b447270929b3a83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7548a96511494929aaebabb4fc679c61"
I20260810 11:45:31.465816 32356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12cca883a699477c8b927c48351a05f9" candidate_uuid: "f2c6cc33ae694e11b447270929b3a83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14289f14acfb44e196d145e10e699131"
I20260810 11:45:31.466050 32356 raft_consensus.cc:3037] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:31.466050 32498 raft_consensus.cc:3037] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:45:31.469416 32498 raft_consensus.cc:2454] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6cc33ae694e11b447270929b3a83b in term 1.
I20260810 11:45:31.469416 32356 raft_consensus.cc:2454] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6cc33ae694e11b447270929b3a83b in term 1.
I20260810 11:45:31.469985 32145 leader_election.cc:302] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14289f14acfb44e196d145e10e699131, 7548a96511494929aaebabb4fc679c61, f2c6cc33ae694e11b447270929b3a83b; no voters:
I20260810 11:45:31.470351 32571 raft_consensus.cc:2781] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:45:31.470880 32571 raft_consensus.cc:686] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 1 LEADER]: Becoming Leader. State: Replica: f2c6cc33ae694e11b447270929b3a83b, State: Running, Role: LEADER
I20260810 11:45:31.471393 32571 consensus_queue.cc:237] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } }
I20260810 11:45:31.476375 32062 catalog_manager.cc:5495] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c6cc33ae694e11b447270929b3a83b (127.20.93.129). New cstate: current_term: 1 leader_uuid: "f2c6cc33ae694e11b447270929b3a83b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c6cc33ae694e11b447270929b3a83b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 } health_report { overall_health: UNKNOWN } } }
W20260810 11:45:31.481844 32547 tablet.cc:2369] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:45:31.508462 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.502701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.20.93.129:46649)
0810 11:45:31.502922 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.502930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.503032 (+ 102us) server_negotiation.cc:407] Connection header received
0810 11:45:31.503311 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.503315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.503420 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.503586 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:45:31.504007 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.504727 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.506167 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.506602 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.506746 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.507301 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:45:31.507820 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:45:31.507993 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.508075 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.508163 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:45:31.517364 32498 raft_consensus.cc:1261] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 1 FOLLOWER]: Refusing update from remote peer f2c6cc33ae694e11b447270929b3a83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:45:31.517364 32356 raft_consensus.cc:1261] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 1 FOLLOWER]: Refusing update from remote peer f2c6cc33ae694e11b447270929b3a83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:45:31.518414 32576 consensus_queue.cc:1035] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "14289f14acfb44e196d145e10e699131" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:45:31.518778 32571 consensus_queue.cc:1035] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7548a96511494929aaebabb4fc679c61" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 35823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:45:31.531958 32579 mvcc.cc:205] Tried to move back new op lower bound from 7316940109883105280 to 7316940109710434304. Current Snapshot: MvccSnapshot[applied={T|T < 7316940109883105280}]
I20260810 11:45:31.531903 32581 mvcc.cc:205] Tried to move back new op lower bound from 7316940109883105280 to 7316940109710434304. Current Snapshot: MvccSnapshot[applied={T|T < 7316940109883105280}]
I20260810 11:45:31.546481 32061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:45:31.546831 32061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:45:31.550808 32061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08ab2f6928b9415f9e4adfe879476ebb: Sending DeleteTablet for 3 replicas of tablet 12cca883a699477c8b927c48351a05f9
I20260810 11:45:31.552147 32195 tablet_service.cc:1514] Processing DeleteTablet for tablet 12cca883a699477c8b927c48351a05f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:45:31 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:48794
I20260810 11:45:31.552306 32334 tablet_service.cc:1514] Processing DeleteTablet for tablet 12cca883a699477c8b927c48351a05f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:45:31 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:48776
I20260810 11:45:31.552407 32477 tablet_service.cc:1514] Processing DeleteTablet for tablet 12cca883a699477c8b927c48351a05f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:45:31 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:36470
I20260810 11:45:31.553203 32590 tablet_replica.cc:330] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: stopping tablet replica
I20260810 11:45:31.553370 32591 tablet_replica.cc:330] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: stopping tablet replica
I20260810 11:45:31.553836 32590 raft_consensus.cc:2229] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:45:31.553876 32591 raft_consensus.cc:2229] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:45:31.554423 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 32125
I20260810 11:45:31.554495 32591 raft_consensus.cc:2258] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:45:31.554807 32590 raft_consensus.cc:2258] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:45:31.557235 32590 ts_tablet_manager.cc:1905] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:45:31.558307 32591 ts_tablet_manager.cc:1905] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:45:31.560425 32592 tablet_replica.cc:330] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: stopping tablet replica
I20260810 11:45:31.565124 32590 ts_tablet_manager.cc:1918] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:45:31.565414 32590 log.cc:1196] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/12cca883a699477c8b927c48351a05f9
I20260810 11:45:31.566167 32590 ts_tablet_manager.cc:1939] T 12cca883a699477c8b927c48351a05f9 P f2c6cc33ae694e11b447270929b3a83b: Deleting consensus metadata
I20260810 11:45:31.566107 32592 raft_consensus.cc:2229] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:45:31.566730 32592 raft_consensus.cc:2258] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:45:31.567597 32591 ts_tablet_manager.cc:1918] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:45:31.567860 32591 log.cc:1196] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/12cca883a699477c8b927c48351a05f9
I20260810 11:45:31.569195 32591 ts_tablet_manager.cc:1939] T 12cca883a699477c8b927c48351a05f9 P 14289f14acfb44e196d145e10e699131: Deleting consensus metadata
I20260810 11:45:31.569561 32592 ts_tablet_manager.cc:1905] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:45:31.571354 32049 catalog_manager.cc:4841] TS 14289f14acfb44e196d145e10e699131 (127.20.93.130:37671): tablet 12cca883a699477c8b927c48351a05f9 (table test-table [id=b589ee50699349fe8509a9005aba240a]) successfully deleted
I20260810 11:45:31.575690 32049 catalog_manager.cc:4841] TS f2c6cc33ae694e11b447270929b3a83b (127.20.93.129:46649): tablet 12cca883a699477c8b927c48351a05f9 (table test-table [id=b589ee50699349fe8509a9005aba240a]) successfully deleted
I20260810 11:45:31.576808 32592 ts_tablet_manager.cc:1918] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:45:31.576984 32592 log.cc:1196] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/12cca883a699477c8b927c48351a05f9
I20260810 11:45:31.577555 32592 ts_tablet_manager.cc:1939] T 12cca883a699477c8b927c48351a05f9 P 7548a96511494929aaebabb4fc679c61: Deleting consensus metadata
I20260810 11:45:31.579653 32048 catalog_manager.cc:4841] TS 7548a96511494929aaebabb4fc679c61 (127.20.93.131:35823): tablet 12cca883a699477c8b927c48351a05f9 (table test-table [id=b589ee50699349fe8509a9005aba240a]) successfully deleted
Aug 10 11:45:31 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:45:31.590095 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.557555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.20.93.129:46649)
0810 11:45:31.557768 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.557775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.568558 (+ 10783us) server_negotiation.cc:407] Connection header received
0810 11:45:31.571726 (+ 3168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.571732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.571826 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.571923 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:31.573985 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.575393 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.577301 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.578096 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.578240 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.583339 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:31.583353 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:31.583367 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:31.583420 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:31.586804 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.587786 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.587794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.587796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.587952 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.588408 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.588411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.588413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.588785 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:31.589007 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.589443 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.589735 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:45:31.590853 32257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:31.692867 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 32125
I20260810 11:45:31.717028 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 32268
Aug 10 11:45:31 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:45:31.731942 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.717976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.20.93.130:37671)
0810 11:45:31.718159 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.718165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.718184 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:45:31.718299 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.718304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.718377 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.718461 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:31.719687 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.720432 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.721710 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.722091 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.722186 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.726294 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:31.726307 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:31.726324 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:31.726358 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:31.729109 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.729881 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.729889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.729891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.730033 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.730469 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.730473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.730475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.730927 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:31.731176 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.731455 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.731664 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:45:31.732750 32399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:31.825928 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 32268
I20260810 11:45:31.845652 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 32411
Aug 10 11:45:31 dist-test-slave-njxd krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:45:31.861334 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:45:31.846527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.20.93.131:35823)
0810 11:45:31.846705 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 11:45:31.846710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:45:31.846745 (+ 35us) server_negotiation.cc:407] Connection header received
0810 11:45:31.846919 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:45:31.846923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:45:31.846999 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:45:31.847110 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:45:31.848881 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.849810 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.851051 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:45:31.851442 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:45:31.851533 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:45:31.855898 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:45:31.855914 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:45:31.855931 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:45:31.855968 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:45:31.858811 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.859488 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.859495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.859497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.859638 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:45:31.859967 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:45:31.859970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:45:31.859972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:45:31.860333 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:45:31.860551 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 11:45:31.860812 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 11:45:31.860978 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:45:31.862102 32541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:31.953491 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 32411
I20260810 11:45:31.972424 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 32028
I20260810 11:45:31.973821 32093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:45:32.085675 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 32028
2026-08-10T11:45:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4617 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:45:32 dist-test-slave-njxd krb5kdc[32601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:45:32 dist-test-slave-njxd krb5kdc[32601](info): set up 2 sockets
Aug 10 11:45:32 dist-test-slave-njxd krb5kdc[32601](info): commencing operation
krb5kdc: starting...
W20260810 11:45:33.261109 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:45:33 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:45:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangeradmin_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangeradmin_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangerlookup_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangerlookup_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
I20260810 11:45:33.684268 20854 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFs48JS/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/postgres -l logfile start
2026-08-10 11:45:36.387 UTC [32633] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 11:45:36.387 UTC [32633] LOG: listening on IPv4 address "127.20.93.148", port 53305
2026-08-10 11:45:36.391 UTC [32633] LOG: listening on Unix socket "/tmp/.s.PGSQL.53305"
2026-08-10 11:45:36.416 UTC [32635] LOG: database system was shut down at 2026-08-10 11:45:34 UTC
2026-08-10 11:45:36.420 UTC [32633] LOG: database system is ready to accept connections
I20260810 11:45:37.449380 20854 mini_postgres.cc:96] Postgres bound to 53305
2026-08-10 11:45:37.475 UTC [32644] FATAL: database "slave" does not exist
127.20.93.148:53305 - accepting connections
I20260810 11:45:37.476542 20854 mini_ranger.cc:161] Starting Ranger...
I20260810 11:45:37.512837 20854 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 11:45:38.024051 20854 mini_ranger.cc:87] Created ranger Postgres database
I20260810 11:45:38.024211 20854 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin
2026-08-10 11:45:38,376 [I] DB FLAVOR :POSTGRES
2026-08-10 11:45:38,376 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:45:38,376 [I] Checking connection..
2026-08-10 11:45:38,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 11:45:38,654 [I] Checking connection passed.
2026-08-10 11:45:38,655 [I] --------- Verifying version history table ---------
2026-08-10 11:45:38,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:45:38,929 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:45:38,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:45:39,178 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:45:39,179 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 11:45:39,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 11:45:39.438 UTC [32711] WARNING: there is no transaction in progress
2026-08-10 11:45:39,445 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 11:45:39,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:45:39,714 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 11:45:39,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:45:39,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 11:45:39,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 11:45:40,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-njxd.c.gcp-upstream.internal','N') ;"
2026-08-10 11:45:40,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:45:40,786 [I] Table x_portal_user does not exist in database ranger
2026-08-10 11:45:40,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:45:41,043 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 11:45:41,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:45:41,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:45:41,552 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 11:45:41,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 11:45:42.605 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.622 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.638 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.654 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.670 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.687 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.743 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.750 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.766 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.776 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.787 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.801 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.811 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.818 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.825 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:42.832 UTC [360] WARNING: there is no transaction in progress
2026-08-10 11:45:43,489 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 11:45:43,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 11:45:43,748 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 11:45:43,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:45:44,013 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 11:45:44,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:45:44,278 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 11:45:44,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:45:44,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:45:44,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-njxd.c.gcp-upstream.internal';"
2026-08-10 11:45:45,000 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 11:45:45,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 11:45:45,247 [I] DB_PATCHES have already been applied
I20260810 11:45:45.253479 20854 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFs48JS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 11:45:45.253639 20854 mini_ranger.cc:193] Using host: 127.20.93.148
I20260810 11:45:45.275439 20854 mini_ranger.cc:227] Ranger admin URL: http://127.20.93.148:54201
Aug 10, 2026 11:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:45:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:45:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 11:45:46.474382 20854 mini_ranger.cc:160] Time spent starting Ranger: real 8.998s user 0.008s sys 0.060s
Aug 10, 2026 11:46:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:46:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 11:46:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:46:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:46:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 11:46:09.329690 20854 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangerkms_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/rangerkms_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.93.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.93.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 11:46:09 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362369, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 11:46:09.557518 20854 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 11:46:09.587220 20854 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 11:46:10.499284 20854 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 11:46:10.499482 20854 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms
I20260810 11:46:10.499604 20854 mini_ranger_kms.cc:227] Using postgres at 127.20.93.148:53305
2026-08-10 11:46:11,003 [I] DB FLAVOR :POSTGRES
2026-08-10 11:46:11,004 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:46:11,004 [I] Checking connection
2026-08-10 11:46:11,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 11:46:11,244 [I] connection success
2026-08-10 11:46:11,244 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 11:46:11,244 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 11:46:11,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 11:46:11,512 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 11:46:11,512 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:46:11,512 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 11:46:11,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:53305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 11:46:11,787 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 11:46:12.292114 20854 mini_ranger_kms.cc:310] Created kms service
I20260810 11:46:12.512995 20854 mini_ranger_kms.cc:326] Created kudu user
I20260810 11:46:12.560712 20854 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 11:46:12 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362369, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:12.995195 20854 mini_ranger_kms.cc:384] Added ranger policy
I20260810 11:46:12.996675 20854 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 11:46:12.996744 20854 mini_ranger_kms.cc:240] Using host: 127.20.93.148
I20260810 11:46:13.022732 20854 mini_ranger_kms.cc:276] Ranger KMS PID: 553
I20260810 11:46:13.022954 20854 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.93.148:45437
Aug 10, 2026 11:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 11:46:15 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786362375, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:46:15.476912 20854 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.919s user 0.143s sys 0.406s
I20260810 11:46:15.477298 20854 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 11:46:15.477540 20854 mini_ranger_kms.cc:401] 127.20.93.148:45437/kms/v1/keys
Aug 10 11:46:15 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362375, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
Aug 10 11:46:15 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362375, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
Aug 10 11:46:18 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:46:18.453747 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40625
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:40625
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:45437/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:46:18.831642 606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:46:18.832129 606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:46:18.832357 606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:46:18.840026 606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:46:18.840148 606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:46:18.840271 606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:46:18.840373 606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:46:18.840474 606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:46:18.851209 606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.93.148:45437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:40625
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:40625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:46:18.852599 606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:46:18.854074 606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:46:18.867729 611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:18.867682 612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:18.867924 614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:46:18.868623 606 server_base.cc:1034] running on GCE node
I20260810 11:46:18.869421 606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:46:18.871093 606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:46:18.872342 606 hybrid_clock.cc:648] HybridClock initialized: now 1786362378872284 us; error 48 us; skew 500 ppm
I20260810 11:46:18.872761 606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:46:18 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:46:18.879148 606 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:46:18.880887 606 webserver.cc:460] Webserver started at http://127.20.93.190:41379/ using document root <none> and password file <none>
I20260810 11:46:18.881696 606 fs_manager.cc:359] Metadata directory not provided
I20260810 11:46:18.881840 606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:46:18.882177 606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:46:18 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:18.937445 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "6d68a194444f46f880c7c8352b4e32f3"
format_stamp: "Formatted at 2026-08-10 11:46:18 on dist-test-slave-njxd"
server_key: "da661ee13ab2317f5cddcc1510a58a18"
server_key_iv: "f068b94b74866d26ff2026f1ead3f5f0"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:18.938656 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "6d68a194444f46f880c7c8352b4e32f3"
format_stamp: "Formatted at 2026-08-10 11:46:18 on dist-test-slave-njxd"
server_key: "da661ee13ab2317f5cddcc1510a58a18"
server_key_iv: "f068b94b74866d26ff2026f1ead3f5f0"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:18.944970 606 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:46:18.966243 621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:18.968215 606 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 11:46:18.968421 606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "6d68a194444f46f880c7c8352b4e32f3"
format_stamp: "Formatted at 2026-08-10 11:46:18 on dist-test-slave-njxd"
server_key: "da661ee13ab2317f5cddcc1510a58a18"
server_key_iv: "f068b94b74866d26ff2026f1ead3f5f0"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:18.968698 606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:46:18.987227 606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:46:18.991426 606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:46:18.991703 606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:46:19.010733 606 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:40625
I20260810 11:46:19.010769 675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:40625 every 8 connection(s)
I20260810 11:46:19.012212 606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:46:19.013859 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 606
I20260810 11:46:19.014073 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:46:19.015008 20854 external_mini_cluster.cc:1428] Setting key f04c34cb10981b5576f7e63f3a8fa032
I20260810 11:46:19.017553 676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:46:19 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:46:19.031560 676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: Bootstrap starting.
I20260810 11:46:19.034929 676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: Neither blocks nor log segments found. Creating new log.
I20260810 11:46:19.036022 676 log.cc:824] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: Log is configured to *not* fsync() on all Append() calls
I20260810 11:46:19.038707 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:19.017294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.20.93.190:40625)
0810 11:46:19.018186 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:19.018198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:19.018242 (+ 44us) server_negotiation.cc:407] Connection header received
0810 11:46:19.019976 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:19.020000 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:19.020408 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:19.020775 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:19.022483 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:19.023915 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:19.025757 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:19.026415 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:19.026541 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:19.031200 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:19.031209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:19.031224 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:19.031262 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:19.034552 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:19.035318 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:19.035325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:19.035332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:19.035495 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:19.035902 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:19.035905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:19.035907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:19.036468 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:19.036837 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:19.037290 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 11:46:19.037748 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":463,"threads_started":1}
I20260810 11:46:19.039247 676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: No bootstrap required, opened a new log
I20260810 11:46:19.047398 676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } }
I20260810 11:46:19.047809 676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:46:19.047924 676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d68a194444f46f880c7c8352b4e32f3, State: Initialized, Role: FOLLOWER
I20260810 11:46:19.048487 676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } }
I20260810 11:46:19.048756 676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:46:19.048906 676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:46:19.049088 676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:46:19.051486 676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } }
I20260810 11:46:19.052093 676 leader_election.cc:302] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d68a194444f46f880c7c8352b4e32f3; no voters:
I20260810 11:46:19.052783 676 leader_election.cc:288] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:46:19.052920 681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:46:19.053699 681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [term 1 LEADER]: Becoming Leader. State: Replica: 6d68a194444f46f880c7c8352b4e32f3, State: Running, Role: LEADER
I20260810 11:46:19.054298 681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } }
I20260810 11:46:19.055084 676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:46:19.057508 683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d68a194444f46f880c7c8352b4e32f3. Latest consensus state: current_term: 1 leader_uuid: "6d68a194444f46f880c7c8352b4e32f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } } }
I20260810 11:46:19.058032 683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [sys.catalog]: This master's current role is: LEADER
I20260810 11:46:19.059556 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d68a194444f46f880c7c8352b4e32f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d68a194444f46f880c7c8352b4e32f3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 40625 } } }
I20260810 11:46:19.059950 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3 [sys.catalog]: This master's current role is: LEADER
I20260810 11:46:19.060519 690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:46:19.066103 690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:46:19.066942 676 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:46:19.092494 690 catalog_manager.cc:1347] Generated new cluster ID: 38091cd731bc43d78420acbcfa25337c
I20260810 11:46:19.092743 690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:46:19.116851 690 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:46:19.118914 690 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:46:19.127415 690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: Generated new TSK 0
I20260810 11:46:19.128341 690 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:46:19.925691 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:45437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:46:20 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 11:46:20.393124 723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:46:20.393471 723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:46:20.393682 723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:46:20.401896 723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:46:20.402112 723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:46:20.402315 723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:46:20.414270 723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.93.148:45437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:46:20.415752 723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:46:20.417446 723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:46:20.436940 734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:20.438146 736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:20.451622 738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:46:20.452394 723 server_base.cc:1034] running on GCE node
I20260810 11:46:20.453389 723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:46:20.454957 723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:46:20.456287 723 hybrid_clock.cc:648] HybridClock initialized: now 1786362380456172 us; error 101 us; skew 500 ppm
I20260810 11:46:20.456779 723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:46:20 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:46:20.465647 723 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:46:20.467882 723 webserver.cc:460] Webserver started at http://127.20.93.129:37085/ using document root <none> and password file <none>
I20260810 11:46:20.468870 723 fs_manager.cc:359] Metadata directory not provided
I20260810 11:46:20.469018 723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:46:20.469422 723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:46:20 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:20.491660 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "cf82936a58ab4a1f87e0748309fc7cc9"
format_stamp: "Formatted at 2026-08-10 11:46:20 on dist-test-slave-njxd"
server_key: "09f3d9e28ef75c7d89228032a0453e16"
server_key_iv: "fdd186539a94a6bce26bdb404c360b31"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:20.493320 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "cf82936a58ab4a1f87e0748309fc7cc9"
format_stamp: "Formatted at 2026-08-10 11:46:20 on dist-test-slave-njxd"
server_key: "09f3d9e28ef75c7d89228032a0453e16"
server_key_iv: "fdd186539a94a6bce26bdb404c360b31"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:20.508085 723 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.005s sys 0.007s
I20260810 11:46:20.534960 745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:20.537216 723 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.004s
I20260810 11:46:20.537534 723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "cf82936a58ab4a1f87e0748309fc7cc9"
format_stamp: "Formatted at 2026-08-10 11:46:20 on dist-test-slave-njxd"
server_key: "09f3d9e28ef75c7d89228032a0453e16"
server_key_iv: "fdd186539a94a6bce26bdb404c360b31"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:20.537940 723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:46:20.560078 723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:46:20.564863 723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:46:20.565304 723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:46:20.566586 723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:46:20.569666 723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:46:20.569850 723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:20.570071 723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:46:20.570302 723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:46:20 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:21.005123 723 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:39005
I20260810 11:46:21.005246 862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:39005 every 8 connection(s)
I20260810 11:46:21.006850 723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:46:21.016203 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 723
I20260810 11:46:21.016413 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:46:21.017258 20854 external_mini_cluster.cc:1428] Setting key 23d9f3c8a4dd7657a308aa188a6f143c
Aug 10 11:46:21 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:46:21.038090 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:21.011795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:60849 (local address 127.20.93.190:40625)
0810 11:46:21.012476 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:21.012483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:21.014688 (+ 2205us) server_negotiation.cc:407] Connection header received
0810 11:46:21.017550 (+ 2862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:21.017556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:21.017676 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:21.017787 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:21.020096 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.021259 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.024226 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.024984 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.025138 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:21.030626 (+ 5488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:21.030633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:21.030638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:21.030692 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:21.034791 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:21.035605 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:21.035611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:21.035615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:21.035772 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:21.036205 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:21.036209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:21.036213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:21.036571 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:21.036716 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:21.037451 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 11:46:21.037721 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":933,"threads_started":1}
I20260810 11:46:21.039340 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:21.011217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:40625 (local address 127.20.93.129:60849)
0810 11:46:21.014480 (+ 3263us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:21.014550 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:21.017067 (+ 2517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:21.018287 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:21.018300 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:21.018968 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:46:21.019863 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:21.019893 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.021464 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.021472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:21.024011 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:21.024055 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.025522 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.025532 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:21.025954 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:21.026849 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:46:21.026886 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:46:21.030300 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:46:21.034957 (+ 4657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:21.034965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:21.034981 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:21.035444 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:21.035882 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:21.035885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:21.035887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:21.036089 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:21.036699 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:46:21.036705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:46:21.037047 (+ 342us) client_negotiation.cc:769] Sending connection context
0810 11:46:21.037285 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 11:46:21.037763 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2944,"thread_start_us":579,"threads_started":1}
I20260810 11:46:21.041200 863 heartbeater.cc:347] Connected to a master server at 127.20.93.190:40625
I20260810 11:46:21.041607 863 heartbeater.cc:464] Registering TS with master...
I20260810 11:46:21.042667 863 heartbeater.cc:511] Master 127.20.93.190:40625 requested a full tablet report, sending...
I20260810 11:46:21.046069 639 ts_manager.cc:194] Registered new tserver with Master: cf82936a58ab4a1f87e0748309fc7cc9 (127.20.93.129:39005)
I20260810 11:46:21.048758 639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:60849
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:46:21.095484 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:45437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:46:21 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362380, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
W20260810 11:46:21.595468 871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:46:21.595798 871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:46:21.595957 871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:46:21.604413 871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:46:21.604558 871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:46:21.604753 871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:46:21.615882 871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.93.148:45437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:46:21.617305 871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:46:21.618750 871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:46:21.636008 890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:21.636241 887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:21.636538 888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:46:21.637549 871 server_base.cc:1034] running on GCE node
I20260810 11:46:21.638671 871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:46:21.640115 871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:46:21.641382 871 hybrid_clock.cc:648] HybridClock initialized: now 1786362381641319 us; error 50 us; skew 500 ppm
I20260810 11:46:21.641844 871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:46:21 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362381, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:46:21.646544 871 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:46:21.648492 871 webserver.cc:460] Webserver started at http://127.20.93.130:46553/ using document root <none> and password file <none>
I20260810 11:46:21.649730 871 fs_manager.cc:359] Metadata directory not provided
I20260810 11:46:21.649884 871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:46:21.650261 871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:46:21 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362381, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:21.672341 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "060bf6fb035648b08af1d37ac1ef93f7"
format_stamp: "Formatted at 2026-08-10 11:46:21 on dist-test-slave-njxd"
server_key: "dcf7288e6511efd1fae89cb614883411"
server_key_iv: "652c8e2c9d7606d4dc661c9b509dc390"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:21.674293 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "060bf6fb035648b08af1d37ac1ef93f7"
format_stamp: "Formatted at 2026-08-10 11:46:21 on dist-test-slave-njxd"
server_key: "dcf7288e6511efd1fae89cb614883411"
server_key_iv: "652c8e2c9d7606d4dc661c9b509dc390"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:21.684007 871 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260810 11:46:21.706399 898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:21.708673 871 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.003s
I20260810 11:46:21.708882 871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "060bf6fb035648b08af1d37ac1ef93f7"
format_stamp: "Formatted at 2026-08-10 11:46:21 on dist-test-slave-njxd"
server_key: "dcf7288e6511efd1fae89cb614883411"
server_key_iv: "652c8e2c9d7606d4dc661c9b509dc390"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:21.709177 871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:46:21.736143 871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:46:21.742332 871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:46:21.742993 871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:46:21.744446 871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:46:21.746582 871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:46:21.746690 871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:21.746876 871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:46:21.746949 871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:21.815470 871 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42715
I20260810 11:46:21.815537 1020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42715 every 8 connection(s)
I20260810 11:46:21.817446 871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:46:21.826606 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 871
I20260810 11:46:21.826786 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:46:21.827329 20854 external_mini_cluster.cc:1428] Setting key f6dd02a44f3bc5fbd0c2b69c3ea21e3b
Aug 10 11:46:21 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362381, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
I20260810 11:46:21.878206 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:21.820813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:56957 (local address 127.20.93.190:40625)
0810 11:46:21.839865 (+ 19052us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:21.839876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:21.841655 (+ 1779us) server_negotiation.cc:407] Connection header received
0810 11:46:21.847303 (+ 5648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:21.847310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:21.847415 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:21.847523 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:21.850100 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.851279 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.854281 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.854976 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.855110 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:21.864207 (+ 9097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:21.864213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:21.864216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:21.864257 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:21.868493 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:21.874158 (+ 5665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:21.874167 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:21.874174 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:21.874543 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:21.875392 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:21.875399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:21.875405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:21.875941 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:21.876101 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:21.877376 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0810 11:46:21.877708 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18806,"thread_start_us":19056,"threads_started":1}
I20260810 11:46:21.880673 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:21.821648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:40625 (local address 127.20.93.130:56957)
0810 11:46:21.840878 (+ 19230us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:21.840957 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:21.846891 (+ 5934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:21.848168 (+ 1277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:21.848181 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:21.848892 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:46:21.849856 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:21.849886 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.851533 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.851538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:21.853935 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:21.854001 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:21.855525 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:21.855533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:21.855841 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:21.856966 (+ 1125us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:46:21.857000 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:46:21.863877 (+ 6877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:46:21.872963 (+ 9086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:21.872980 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:21.873004 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:21.873964 (+ 960us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:21.874785 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:21.874792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:21.874798 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:21.875242 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:21.876261 (+ 1019us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:46:21.876273 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:46:21.876781 (+ 508us) client_negotiation.cc:769] Sending connection context
0810 11:46:21.878455 (+ 1674us) client_negotiation.cc:240] Negotiation successful
0810 11:46:21.879091 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18722,"thread_start_us":19820,"threads_started":1}
I20260810 11:46:21.882794 1021 heartbeater.cc:347] Connected to a master server at 127.20.93.190:40625
I20260810 11:46:21.883134 1021 heartbeater.cc:464] Registering TS with master...
I20260810 11:46:21.884048 1021 heartbeater.cc:511] Master 127.20.93.190:40625 requested a full tablet report, sending...
I20260810 11:46:21.886025 639 ts_manager.cc:194] Registered new tserver with Master: 060bf6fb035648b08af1d37ac1ef93f7 (127.20.93.130:42715)
I20260810 11:46:21.887373 639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:56957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:46:21.905144 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:45437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:46:22.053428 863 heartbeater.cc:503] Master 127.20.93.190:40625 was elected leader, sending a full tablet report...
W20260810 11:46:22.301903 1027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:46:22.302206 1027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:46:22.302309 1027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:46:22.309950 1027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:46:22.310091 1027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:46:22.310218 1027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:46:22.321267 1027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.93.148:45437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:40625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:46:22.322682 1027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:46:22.323995 1027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:46:22.338316 1033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:22.338483 1035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:46:22.338886 1032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:46:22.339242 1027 server_base.cc:1034] running on GCE node
I20260810 11:46:22.339879 1027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:46:22.340925 1027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:46:22.342191 1027 hybrid_clock.cc:648] HybridClock initialized: now 1786362382342132 us; error 49 us; skew 500 ppm
I20260810 11:46:22.342564 1027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:46:22 dist-test-slave-njxd krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362382, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:46:22.346202 1027 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:46:22.347718 1027 webserver.cc:460] Webserver started at http://127.20.93.131:43933/ using document root <none> and password file <none>
I20260810 11:46:22.348469 1027 fs_manager.cc:359] Metadata directory not provided
I20260810 11:46:22.348616 1027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:46:22.348927 1027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:46:22 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362382, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:46:22.366350 1027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "6c109ba5bb7a40a38802b37a2e65992d"
format_stamp: "Formatted at 2026-08-10 11:46:22 on dist-test-slave-njxd"
server_key: "cf2f37f2efe4d907b88781a56addfec9"
server_key_iv: "edcf3d2adb366a050c45094445cd0cd8"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:22.367223 1027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "6c109ba5bb7a40a38802b37a2e65992d"
format_stamp: "Formatted at 2026-08-10 11:46:22 on dist-test-slave-njxd"
server_key: "cf2f37f2efe4d907b88781a56addfec9"
server_key_iv: "edcf3d2adb366a050c45094445cd0cd8"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:22.372819 1027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:46:22.391713 1042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:22.393944 1027 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:46:22.394212 1027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "6c109ba5bb7a40a38802b37a2e65992d"
format_stamp: "Formatted at 2026-08-10 11:46:22 on dist-test-slave-njxd"
server_key: "cf2f37f2efe4d907b88781a56addfec9"
server_key_iv: "edcf3d2adb366a050c45094445cd0cd8"
server_key_version: "kuduclusterkey@0"
I20260810 11:46:22.394520 1027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:46:22.414278 1027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:46:22.418423 1027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:46:22.418901 1027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:46:22.420048 1027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:46:22.421463 1027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:46:22.421566 1027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:22.421722 1027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:46:22.421797 1027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:46:22.456570 1027 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41153
I20260810 11:46:22.456609 1160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41153 every 8 connection(s)
I20260810 11:46:22.458065 1027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:46:22.467399 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 1027
I20260810 11:46:22.467578 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:46:22.468042 20854 external_mini_cluster.cc:1428] Setting key e5051dd8c5cef32d92adab8f40f7d4e3
Aug 10 11:46:22 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362382, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:46:22.478703 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.460801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:57737 (local address 127.20.93.190:40625)
0810 11:46:22.461322 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.461327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.462200 (+ 873us) server_negotiation.cc:407] Connection header received
0810 11:46:22.464025 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.464031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.464149 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.464278 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:22.465862 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.466619 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.468366 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.468898 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.469027 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.472908 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:22.472912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:22.472916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:22.472949 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:22.475897 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.476618 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.476622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.476624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.476767 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.477150 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.477153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.477155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.477437 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:22.477556 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.478243 (+ 687us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.478435 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":279,"threads_started":1}
I20260810 11:46:22.479727 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.461258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:40625 (local address 127.20.93.131:57737)
0810 11:46:22.462025 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.462071 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.463732 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.464544 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.464552 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.465009 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:46:22.465680 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.465700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.466798 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.466802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.468223 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.468236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.469173 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.469181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.469530 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.470292 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:46:22.470316 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:46:22.472701 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:46:22.476033 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:22.476040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:22.476054 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:22.476507 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:22.476857 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:46:22.476860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:46:22.476862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:46:22.477054 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:46:22.477538 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:46:22.477544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:46:22.477919 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.478148 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.478533 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":328,"threads_started":1}
I20260810 11:46:22.481153 1161 heartbeater.cc:347] Connected to a master server at 127.20.93.190:40625
I20260810 11:46:22.481640 1161 heartbeater.cc:464] Registering TS with master...
I20260810 11:46:22.482596 1161 heartbeater.cc:511] Master 127.20.93.190:40625 requested a full tablet report, sending...
I20260810 11:46:22.484122 639 ts_manager.cc:194] Registered new tserver with Master: 6c109ba5bb7a40a38802b37a2e65992d (127.20.93.131:41153)
I20260810 11:46:22.484959 639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:57737
I20260810 11:46:22.496351 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:46:22.514827 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.501389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.20.93.190:40625)
0810 11:46:22.501552 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.501572 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.502119 (+ 547us) server_negotiation.cc:407] Connection header received
0810 11:46:22.502339 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.502343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.502423 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.502526 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:22.504128 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.504895 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.506369 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.506810 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.506953 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.509342 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:22.509348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:22.509351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:22.509387 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:22.512010 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.512714 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.512721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.512724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.512872 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.513277 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.513283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.513287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.513567 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:22.513886 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.514226 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.514478 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:46:22.519268 639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:46:22.526883 639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:46:22.564011 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.539732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.20.93.131:41153)
0810 11:46:22.553029 (+ 13297us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.553038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.553075 (+ 37us) server_negotiation.cc:407] Connection header received
0810 11:46:22.553175 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.553185 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.553520 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.553820 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:46:22.555275 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.557067 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.561004 (+ 3937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.562664 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.562832 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.563091 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.563434 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.563610 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13102,"thread_start_us":12955,"threads_started":1}
I20260810 11:46:22.564029 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.539515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41153 (local address 127.0.0.1:48790)
0810 11:46:22.550410 (+ 10895us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.550482 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.550741 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.554249 (+ 3508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.554261 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.554307 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:46:22.554921 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.554938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.557285 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.557294 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.560234 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.560250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.562893 (+ 2643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.562900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.563158 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.563206 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.563385 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.563618 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10518,"thread_start_us":21439,"threads_started":1}
I20260810 11:46:22.567607 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.539582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:39005 (local address 127.0.0.1:41344)
0810 11:46:22.552235 (+ 12653us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.552265 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.552406 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.560384 (+ 7978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.560388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.560419 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:46:22.560875 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.560892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.562921 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.562926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.564974 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.564987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.566740 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.566747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.566937 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.566973 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.567073 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.567210 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12490,"thread_start_us":12339,"threads_started":1}
I20260810 11:46:22.567667 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.540255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.20.93.129:39005)
0810 11:46:22.559463 (+ 19208us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.559477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.559519 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:46:22.559656 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.559669 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.560057 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.560342 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:46:22.561144 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.562762 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.565138 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.566501 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.566645 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.566918 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.567135 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.567303 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18940,"thread_start_us":20338,"threads_started":1}
I20260810 11:46:22.570611 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.540308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.20.93.130:42715)
0810 11:46:22.560783 (+ 20475us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.560793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.561429 (+ 636us) server_negotiation.cc:407] Connection header received
0810 11:46:22.561632 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.561642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.562002 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.562276 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:46:22.562903 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.565779 (+ 2876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.567940 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.569421 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.569598 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.569810 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.570231 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.570371 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18674,"thread_start_us":20184,"threads_started":1}
I20260810 11:46:22.570662 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.540231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42715 (local address 127.0.0.1:45502)
0810 11:46:22.561280 (+ 21049us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.561301 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.561438 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.562265 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.562270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.562297 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:46:22.562661 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.562675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.565935 (+ 3260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.565941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.567736 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.567754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.569844 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.569853 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.570047 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.570079 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.570195 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.570339 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20911,"thread_start_us":10757,"threads_started":1}
I20260810 11:46:22.572127 794 tablet_service.cc:1467] Processing CreateTablet for tablet c9a2512eb12745f692e0035ab4868bfb (DEFAULT_TABLE table=test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:46:22.569001 1094 tablet_service.cc:1467] Processing CreateTablet for tablet c9a2512eb12745f692e0035ab4868bfb (DEFAULT_TABLE table=test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:46:22.573640 794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a2512eb12745f692e0035ab4868bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:46:22.574321 1094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a2512eb12745f692e0035ab4868bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:46:22.574358 947 tablet_service.cc:1467] Processing CreateTablet for tablet c9a2512eb12745f692e0035ab4868bfb (DEFAULT_TABLE table=test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:46:22.575776 947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9a2512eb12745f692e0035ab4868bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:46:22.585964 1182 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Bootstrap starting.
I20260810 11:46:22.589504 1182 tablet_bootstrap.cc:654] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Neither blocks nor log segments found. Creating new log.
I20260810 11:46:22.590442 1183 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Bootstrap starting.
I20260810 11:46:22.590993 1182 log.cc:824] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:46:22.592279 1184 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Bootstrap starting.
I20260810 11:46:22.595619 1184 tablet_bootstrap.cc:654] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Neither blocks nor log segments found. Creating new log.
I20260810 11:46:22.595741 1183 tablet_bootstrap.cc:654] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Neither blocks nor log segments found. Creating new log.
I20260810 11:46:22.596776 1182 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: No bootstrap required, opened a new log
I20260810 11:46:22.596745 1184 log.cc:824] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:46:22.597102 1182 ts_tablet_manager.cc:1397] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260810 11:46:22.597085 1183 log.cc:824] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:46:22.600098 1183 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: No bootstrap required, opened a new log
I20260810 11:46:22.600410 1184 tablet_bootstrap.cc:492] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: No bootstrap required, opened a new log
I20260810 11:46:22.600440 1183 ts_tablet_manager.cc:1397] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260810 11:46:22.600677 1184 ts_tablet_manager.cc:1397] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 11:46:22.608361 1182 raft_consensus.cc:348] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.608544 1183 raft_consensus.cc:348] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.608870 1182 raft_consensus.cc:374] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:46:22.608902 1183 raft_consensus.cc:374] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:46:22.608989 1183 raft_consensus.cc:729] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c109ba5bb7a40a38802b37a2e65992d, State: Initialized, Role: FOLLOWER
I20260810 11:46:22.609012 1182 raft_consensus.cc:729] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf82936a58ab4a1f87e0748309fc7cc9, State: Initialized, Role: FOLLOWER
I20260810 11:46:22.609661 1183 consensus_queue.cc:260] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.609738 1182 consensus_queue.cc:260] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.610896 1161 heartbeater.cc:503] Master 127.20.93.190:40625 was elected leader, sending a full tablet report...
I20260810 11:46:22.611459 1183 ts_tablet_manager.cc:1428] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 11:46:22.614614 1184 raft_consensus.cc:348] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.615362 1184 raft_consensus.cc:374] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:46:22.615533 1184 raft_consensus.cc:729] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060bf6fb035648b08af1d37ac1ef93f7, State: Initialized, Role: FOLLOWER
I20260810 11:46:22.616451 1184 consensus_queue.cc:260] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.619899 1182 ts_tablet_manager.cc:1428] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Time spent starting tablet: real 0.023s user 0.020s sys 0.002s
I20260810 11:46:22.620995 1021 heartbeater.cc:503] Master 127.20.93.190:40625 was elected leader, sending a full tablet report...
I20260810 11:46:22.621624 1184 ts_tablet_manager.cc:1428] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
I20260810 11:46:22.639055 1189 raft_consensus.cc:482] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:46:22.639377 1189 raft_consensus.cc:504] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.641140 1189 leader_election.cc:288] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c109ba5bb7a40a38802b37a2e65992d (127.20.93.131:41153), 060bf6fb035648b08af1d37ac1ef93f7 (127.20.93.130:42715)
I20260810 11:46:22.648000 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.641500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:58767 (local address 127.20.93.131:41153)
0810 11:46:22.642085 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.642094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.642373 (+ 279us) server_negotiation.cc:407] Connection header received
0810 11:46:22.642635 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.642645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.642710 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.642804 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:46:22.643363 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.644125 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.646235 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.647052 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.647238 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.647316 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.647601 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.647727 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417}
I20260810 11:46:22.648092 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.641715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41651 (local address 127.20.93.130:42715)
0810 11:46:22.641960 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.641967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.642673 (+ 706us) server_negotiation.cc:407] Connection header received
0810 11:46:22.642874 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.642878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.642951 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.643049 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:46:22.643660 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.644423 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.646353 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.647134 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.647302 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.647394 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.647690 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.647828 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:46:22.648221 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.641474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41153 (local address 127.20.93.129:58767)
0810 11:46:22.642253 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.642271 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.642465 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.642823 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.642828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.642855 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:46:22.643226 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.643235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.644245 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.644252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.646068 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.646082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.647263 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.647269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.647435 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.647459 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.647575 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.647718 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":59,"thread_start_us":429,"threads_started":1}
I20260810 11:46:22.648407 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.641608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42715 (local address 127.20.93.129:41651)
0810 11:46:22.642563 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0810 11:46:22.642578 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:46:22.642732 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:46:22.643067 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:46:22.643071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:46:22.643098 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:46:22.643489 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.643504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.644576 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.644584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.646196 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:46:22.646209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.647386 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.647393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:46:22.647527 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.647550 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:46:22.647651 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 11:46:22.647783 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":2,"thread_start_us":576,"threads_started":1}
I20260810 11:46:22.649387 971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a2512eb12745f692e0035ab4868bfb" candidate_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060bf6fb035648b08af1d37ac1ef93f7" is_pre_election: true
I20260810 11:46:22.649387 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a2512eb12745f692e0035ab4868bfb" candidate_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c109ba5bb7a40a38802b37a2e65992d" is_pre_election: true
I20260810 11:46:22.650043 971 raft_consensus.cc:2454] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf82936a58ab4a1f87e0748309fc7cc9 in term 0.
I20260810 11:46:22.650051 1115 raft_consensus.cc:2454] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf82936a58ab4a1f87e0748309fc7cc9 in term 0.
I20260810 11:46:22.650936 747 leader_election.cc:302] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c109ba5bb7a40a38802b37a2e65992d, cf82936a58ab4a1f87e0748309fc7cc9; no voters:
I20260810 11:46:22.651422 1189 raft_consensus.cc:2781] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:46:22.651608 1189 raft_consensus.cc:482] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:46:22.651736 1189 raft_consensus.cc:3037] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:46:22.654294 1189 raft_consensus.cc:504] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.654945 1189 leader_election.cc:288] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [CANDIDATE]: Term 1 election: Requested vote from peers 6c109ba5bb7a40a38802b37a2e65992d (127.20.93.131:41153), 060bf6fb035648b08af1d37ac1ef93f7 (127.20.93.130:42715)
I20260810 11:46:22.655443 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a2512eb12745f692e0035ab4868bfb" candidate_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c109ba5bb7a40a38802b37a2e65992d"
I20260810 11:46:22.655570 971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9a2512eb12745f692e0035ab4868bfb" candidate_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060bf6fb035648b08af1d37ac1ef93f7"
I20260810 11:46:22.655689 1115 raft_consensus.cc:3037] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:46:22.655813 971 raft_consensus.cc:3037] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:46:22.658350 1115 raft_consensus.cc:2454] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf82936a58ab4a1f87e0748309fc7cc9 in term 1.
I20260810 11:46:22.658357 971 raft_consensus.cc:2454] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf82936a58ab4a1f87e0748309fc7cc9 in term 1.
I20260810 11:46:22.658814 747 leader_election.cc:302] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c109ba5bb7a40a38802b37a2e65992d, cf82936a58ab4a1f87e0748309fc7cc9; no voters:
I20260810 11:46:22.659147 1189 raft_consensus.cc:2781] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:46:22.659755 1189 raft_consensus.cc:686] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 1 LEADER]: Becoming Leader. State: Replica: cf82936a58ab4a1f87e0748309fc7cc9, State: Running, Role: LEADER
I20260810 11:46:22.660259 1189 consensus_queue.cc:237] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } }
I20260810 11:46:22.665446 639 catalog_manager.cc:5495] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf82936a58ab4a1f87e0748309fc7cc9 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf82936a58ab4a1f87e0748309fc7cc9" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 39005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 } health_report { overall_health: UNKNOWN } } }
I20260810 11:46:22.689369 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.683876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.20.93.129:39005)
0810 11:46:22.684053 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.684057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.684075 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:46:22.684265 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.684268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.684348 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.684480 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:46:22.684929 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.685715 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.687256 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.687649 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.687745 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.688231 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:46:22.688793 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:46:22.688905 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.689052 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.689131 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 11:46:22.697971 971 raft_consensus.cc:1261] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 1 FOLLOWER]: Refusing update from remote peer cf82936a58ab4a1f87e0748309fc7cc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:46:22.698050 1115 raft_consensus.cc:1261] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 1 FOLLOWER]: Refusing update from remote peer cf82936a58ab4a1f87e0748309fc7cc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:46:22.699085 1189 consensus_queue.cc:1035] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "060bf6fb035648b08af1d37ac1ef93f7" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:46:22.699481 1193 consensus_queue.cc:1035] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c109ba5bb7a40a38802b37a2e65992d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:46:22.709966 1162 tablet.cc:2369] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:46:22.711650 1197 mvcc.cc:205] Tried to move back new op lower bound from 7316940319520124928 to 7316940319380320256. Current Snapshot: MvccSnapshot[applied={T|T < 7316940319520124928}]
I20260810 11:46:22.726012 638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:46:22.726378 638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:46:22.730590 638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d68a194444f46f880c7c8352b4e32f3: Sending DeleteTablet for 3 replicas of tablet c9a2512eb12745f692e0035ab4868bfb
I20260810 11:46:22.731861 1094 tablet_service.cc:1514] Processing DeleteTablet for tablet c9a2512eb12745f692e0035ab4868bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:46:22 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:48790
I20260810 11:46:22.731861 794 tablet_service.cc:1514] Processing DeleteTablet for tablet c9a2512eb12745f692e0035ab4868bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:46:22 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:41344
I20260810 11:46:22.731978 947 tablet_service.cc:1514] Processing DeleteTablet for tablet c9a2512eb12745f692e0035ab4868bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:46:22 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45502
I20260810 11:46:22.734310 1208 tablet_replica.cc:330] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: stopping tablet replica
I20260810 11:46:22.734846 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 723
I20260810 11:46:22.734871 1208 raft_consensus.cc:2229] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:46:22.735417 1208 raft_consensus.cc:2258] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:46:22.739253 1208 ts_tablet_manager.cc:1905] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:46:22.748950 1207 tablet_replica.cc:330] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: stopping tablet replica
I20260810 11:46:22.749765 1207 raft_consensus.cc:2229] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:46:22.750545 1207 raft_consensus.cc:2258] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:46:22.751566 1206 tablet_replica.cc:330] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: stopping tablet replica
I20260810 11:46:22.753607 1207 ts_tablet_manager.cc:1905] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:46:22.754149 1206 raft_consensus.cc:2229] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:46:22.754979 1208 ts_tablet_manager.cc:1918] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:46:22.755219 1208 log.cc:1196] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/c9a2512eb12745f692e0035ab4868bfb
I20260810 11:46:22.755853 1208 ts_tablet_manager.cc:1939] T c9a2512eb12745f692e0035ab4868bfb P 060bf6fb035648b08af1d37ac1ef93f7: Deleting consensus metadata
I20260810 11:46:22.757953 623 catalog_manager.cc:4841] TS 060bf6fb035648b08af1d37ac1ef93f7 (127.20.93.130:42715): tablet c9a2512eb12745f692e0035ab4868bfb (table test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]) successfully deleted
I20260810 11:46:22.758358 1206 raft_consensus.cc:2258] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:46:22.760425 1206 ts_tablet_manager.cc:1905] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:46:22.762149 1207 ts_tablet_manager.cc:1918] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:46:22.762405 1207 log.cc:1196] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/c9a2512eb12745f692e0035ab4868bfb
I20260810 11:46:22.762957 1207 ts_tablet_manager.cc:1939] T c9a2512eb12745f692e0035ab4868bfb P 6c109ba5bb7a40a38802b37a2e65992d: Deleting consensus metadata
I20260810 11:46:22.764973 624 catalog_manager.cc:4841] TS 6c109ba5bb7a40a38802b37a2e65992d (127.20.93.131:41153): tablet c9a2512eb12745f692e0035ab4868bfb (table test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]) successfully deleted
Aug 10 11:46:22 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:46:22.767745 1206 ts_tablet_manager.cc:1918] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:46:22.767944 1206 log.cc:1196] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/c9a2512eb12745f692e0035ab4868bfb
I20260810 11:46:22.768646 1206 ts_tablet_manager.cc:1939] T c9a2512eb12745f692e0035ab4868bfb P cf82936a58ab4a1f87e0748309fc7cc9: Deleting consensus metadata
I20260810 11:46:22.770761 626 catalog_manager.cc:4841] TS cf82936a58ab4a1f87e0748309fc7cc9 (127.20.93.129:39005): tablet c9a2512eb12745f692e0035ab4868bfb (table test-table [id=5d1553181b4b46dc9cb7780a4b3c383f]) successfully deleted
I20260810 11:46:22.775095 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.736543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.20.93.129:39005)
0810 11:46:22.736724 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.736730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.754181 (+ 17451us) server_negotiation.cc:407] Connection header received
0810 11:46:22.754445 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.754451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.754557 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.754648 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:22.757039 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.758318 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.760825 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.761522 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.761700 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.767696 (+ 5996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:22.767717 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:22.767746 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:22.767789 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:22.771800 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.772731 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.772738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.772741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.772890 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.773457 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.773461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.773463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.773897 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:22.774221 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.774604 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.774789 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:46:22.775784 858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:46:22.890260 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 723
I20260810 11:46:22.912528 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 871
Aug 10 11:46:22 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:46:22.927481 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:22.913355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.20.93.130:42715)
0810 11:46:22.913616 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:22.913624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:22.913688 (+ 64us) server_negotiation.cc:407] Connection header received
0810 11:46:22.913853 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:22.913857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:22.913934 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:22.914022 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:22.915446 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.916225 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.917478 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:22.917964 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:22.918054 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:22.922330 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:22.922342 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:22.922357 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:22.922389 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:22.925083 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.925779 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.925787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.925789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.925911 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:22.926248 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:22.926252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:22.926254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:22.926612 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:22.926824 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:22.927066 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 11:46:22.927229 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:46:22.928133 1016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:46:23.017884 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 871
I20260810 11:46:23.037976 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 1027
Aug 10 11:46:23 dist-test-slave-njxd krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:46:23.053680 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:46:23.038796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48802 (local address 127.20.93.131:41153)
0810 11:46:23.038986 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:46:23.038990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:46:23.039010 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:46:23.039194 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:46:23.039199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:46:23.039276 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:46:23.039375 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:46:23.040835 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:23.042030 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:23.043299 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:46:23.043674 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:46:23.043770 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:46:23.048385 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:46:23.048401 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:46:23.048417 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:46:23.048452 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:46:23.051272 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:23.051879 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:23.051885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:23.051888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:23.052003 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:46:23.052328 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:46:23.052331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:46:23.052333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:46:23.052714 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:46:23.052931 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 11:46:23.053178 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 11:46:23.053345 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:46:23.054387 1156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:46:23.149613 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 1027
I20260810 11:46:23.170603 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 606
I20260810 11:46:23.171872 671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:46:23.286865 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 606
I20260810 11:46:23.315858 20854 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 11:46:23.661830 20854 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 11:46:23.662169 20854 mini_ranger.cc:66] Stopping Ranger...
I20260810 11:46:24.008570 20854 mini_ranger.cc:68] Stopped Ranger
2026-08-10 11:46:24.008 UTC [32633] LOG: received smart shutdown request
2026-08-10 11:46:24.012 UTC [32633] LOG: background worker "logical replication launcher" (PID 32642) exited with exit code 1
2026-08-10 11:46:24.013 UTC [32637] LOG: shutting down
2026-08-10 11:46:24.182 UTC [32633] LOG: database system is shut down
2026-08-10T11:46:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52227 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:46:24 dist-test-slave-njxd krb5kdc[1223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:46:24 dist-test-slave-njxd krb5kdc[1223](info): set up 2 sockets
Aug 10 11:46:24 dist-test-slave-njxd krb5kdc[1223](info): commencing operation
krb5kdc: starting...
W20260810 11:46:25.497159 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.011s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:46:25 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:46:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangeradmin_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangeradmin_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangerlookup_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangerlookup_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
I20260810 11:46:25.887918 20854 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFs48JS/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/postgres -l logfile start
2026-08-10 11:46:28.350 UTC [1256] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 11:46:28.350 UTC [1256] LOG: listening on IPv4 address "127.20.93.148", port 32929
2026-08-10 11:46:28.354 UTC [1256] LOG: listening on Unix socket "/tmp/.s.PGSQL.32929"
2026-08-10 11:46:28.378 UTC [1258] LOG: database system was shut down at 2026-08-10 11:46:26 UTC
2026-08-10 11:46:28.382 UTC [1256] LOG: database system is ready to accept connections
I20260810 11:46:29.421881 20854 mini_postgres.cc:96] Postgres bound to 32929
2026-08-10 11:46:29.446 UTC [1267] FATAL: database "slave" does not exist
127.20.93.148:32929 - accepting connections
I20260810 11:46:29.447182 20854 mini_ranger.cc:161] Starting Ranger...
I20260810 11:46:29.480269 20854 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 11:46:29.964646 20854 mini_ranger.cc:87] Created ranger Postgres database
I20260810 11:46:29.964783 20854 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin
2026-08-10 11:46:30,262 [I] DB FLAVOR :POSTGRES
2026-08-10 11:46:30,262 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:46:30,262 [I] Checking connection..
2026-08-10 11:46:30,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 11:46:30,513 [I] Checking connection passed.
2026-08-10 11:46:30,513 [I] --------- Verifying version history table ---------
2026-08-10 11:46:30,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:46:30,804 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:46:30,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:46:31,083 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:46:31,084 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 11:46:31,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 11:46:31.352 UTC [1340] WARNING: there is no transaction in progress
2026-08-10 11:46:31,360 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 11:46:31,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:46:31,632 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 11:46:31,633 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:46:31,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 11:46:31,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 11:46:32,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-njxd.c.gcp-upstream.internal','N') ;"
2026-08-10 11:46:32,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:46:32,700 [I] Table x_portal_user does not exist in database ranger
2026-08-10 11:46:32,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:46:32,967 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 11:46:32,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:46:33,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:46:33,494 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 11:46:33,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 11:46:34.568 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.585 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.601 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.616 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.633 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.654 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.721 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.728 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.743 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.754 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.765 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.780 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.790 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.797 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.804 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:34.811 UTC [1457] WARNING: there is no transaction in progress
2026-08-10 11:46:35,524 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 11:46:35,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 11:46:35,780 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 11:46:35,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:46:36,044 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 11:46:36,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:46:36,325 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 11:46:36,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:46:36,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:46:36,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-njxd.c.gcp-upstream.internal';"
2026-08-10 11:46:37,108 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 11:46:37,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 11:46:37,421 [I] DB_PATCHES have already been applied
I20260810 11:46:37.430729 20854 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFs48JS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 11:46:37.430851 20854 mini_ranger.cc:193] Using host: 127.20.93.148
I20260810 11:46:37.461843 20854 mini_ranger.cc:227] Ranger admin URL: http://127.20.93.148:49815
Aug 10, 2026 11:46:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:46:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:46:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:46:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:46:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:46:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:46:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 11:46:38.959419 20854 mini_ranger.cc:160] Time spent starting Ranger: real 9.512s user 0.004s sys 0.099s
Aug 10, 2026 11:47:00 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:47:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 11:47:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:47:00 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:47:01 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 11:47:02.006096 20854 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.93.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.93.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangerkms_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.93.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/rangerkms_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.93.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.93.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.93.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/HTTP_127.20.93.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 11:47:02 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362422, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 11:47:02.243985 20854 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 11:47:02.272294 20854 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 11:47:03.304020 20854 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 11:47:03.304148 20854 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms
I20260810 11:47:03.304203 20854 mini_ranger_kms.cc:227] Using postgres at 127.20.93.148:32929
2026-08-10 11:47:03,686 [I] DB FLAVOR :POSTGRES
2026-08-10 11:47:03,687 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:47:03,687 [I] Checking connection
2026-08-10 11:47:03,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 11:47:03,945 [I] connection success
2026-08-10 11:47:03,945 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 11:47:03,945 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 11:47:03,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 11:47:04,231 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 11:47:04,231 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:47:04,231 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 11:47:04,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.93.148:32929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 11:47:04,543 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 11:47:05.064244 20854 mini_ranger_kms.cc:310] Created kms service
I20260810 11:47:05.286899 20854 mini_ranger_kms.cc:326] Created kudu user
I20260810 11:47:05.355979 20854 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 11:47:05 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362422, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:05.850883 20854 mini_ranger_kms.cc:384] Added ranger policy
I20260810 11:47:05.851231 20854 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFs48JS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFs48JS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 11:47:05.851285 20854 mini_ranger_kms.cc:240] Using host: 127.20.93.148
I20260810 11:47:05.874107 20854 mini_ranger_kms.cc:276] Ranger KMS PID: 1653
I20260810 11:47:05.874284 20854 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.93.148:50457
Aug 10, 2026 11:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:47:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:47:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:47:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 11:47:08 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786362428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:08.258584 20854 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.015s user 0.061s sys 0.358s
I20260810 11:47:08.258842 20854 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 11:47:08.259028 20854 mini_ranger_kms.cc:401] 127.20.93.148:50457/kms/v1/keys
Aug 10 11:47:08 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
Aug 10 11:47:08 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.93.148@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
Aug 10 11:47:11 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:47:11.459198 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35501
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35501
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:11.827369 1708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:11.827766 1708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:11.827931 1708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:11.835646 1708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:11.835790 1708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:11.835875 1708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:11.835968 1708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:11.836061 1708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:11.847187 1708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.93.148:50457/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35501
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:11.848663 1708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:11.850262 1708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:11.863730 1713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:11.863745 1714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:11.863751 1716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:11.864053 1708 server_base.cc:1034] running on GCE node
I20260810 11:47:11.865157 1708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:11.866991 1708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:11.868304 1708 hybrid_clock.cc:648] HybridClock initialized: now 1786362431868242 us; error 44 us; skew 500 ppm
I20260810 11:47:11.868726 1708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:11 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:11.872397 1708 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:11.874323 1708 webserver.cc:460] Webserver started at http://127.20.93.190:41711/ using document root <none> and password file <none>
I20260810 11:47:11.875118 1708 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:11.875283 1708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:11.875630 1708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:47:11 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:11.929086 1708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "b799cd9aa9be424fa4d856cf0a1f16ae"
format_stamp: "Formatted at 2026-08-10 11:47:11 on dist-test-slave-njxd"
server_key: "7303bb1a7b15edb5e42aa2bea92e9b4f"
server_key_iv: "8ed44997bc90e59d8acbf874d19740ba"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:11.930315 1708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "b799cd9aa9be424fa4d856cf0a1f16ae"
format_stamp: "Formatted at 2026-08-10 11:47:11 on dist-test-slave-njxd"
server_key: "7303bb1a7b15edb5e42aa2bea92e9b4f"
server_key_iv: "8ed44997bc90e59d8acbf874d19740ba"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:11.936690 1708 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:47:11.954401 1723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:11.956171 1708 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:47:11.956377 1708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "b799cd9aa9be424fa4d856cf0a1f16ae"
format_stamp: "Formatted at 2026-08-10 11:47:11 on dist-test-slave-njxd"
server_key: "7303bb1a7b15edb5e42aa2bea92e9b4f"
server_key_iv: "8ed44997bc90e59d8acbf874d19740ba"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:11.956630 1708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:11.975495 1708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:11.980055 1708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:11.980418 1708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:12.001492 1708 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35501
I20260810 11:47:12.001513 1778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35501 every 8 connection(s)
I20260810 11:47:12.003271 1708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:12.007539 1779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:12.013561 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 1708
I20260810 11:47:12.013861 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:47:12.014598 20854 external_mini_cluster.cc:1428] Setting key 59299130513fc79fce0088948304b165
I20260810 11:47:12.020105 1779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Bootstrap starting.
I20260810 11:47:12.024227 1779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:12.025800 1779 log.cc:824] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:12.029964 1779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: No bootstrap required, opened a new log
Aug 10 11:47:12 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:12.038486 1779 raft_consensus.cc:348] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:12.038938 1779 raft_consensus.cc:374] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:12.039079 1779 raft_consensus.cc:729] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b799cd9aa9be424fa4d856cf0a1f16ae, State: Initialized, Role: FOLLOWER
I20260810 11:47:12.039827 1779 consensus_queue.cc:260] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:12.040161 1779 raft_consensus.cc:388] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:12.040329 1779 raft_consensus.cc:482] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:12.040530 1779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:12.041497 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:12.016873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.20.93.190:35501)
0810 11:47:12.017896 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:12.017911 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:12.017958 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:47:12.019995 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:12.020023 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:12.020551 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:12.020937 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:12.022967 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:12.024467 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:12.026613 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:12.027793 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:12.028024 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:12.032985 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:12.032998 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:12.033018 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:12.033079 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:12.036547 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:12.037394 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:12.037402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:12.037410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:12.037666 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:12.038101 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:12.038113 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:12.038116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:12.038725 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:12.039070 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:12.039823 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 11:47:12.040312 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":473,"threads_started":1}
I20260810 11:47:12.043607 1779 raft_consensus.cc:504] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:12.044180 1779 leader_election.cc:302] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b799cd9aa9be424fa4d856cf0a1f16ae; no voters:
I20260810 11:47:12.044865 1779 leader_election.cc:288] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:47:12.045008 1784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:12.045979 1784 raft_consensus.cc:686] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 LEADER]: Becoming Leader. State: Replica: b799cd9aa9be424fa4d856cf0a1f16ae, State: Running, Role: LEADER
I20260810 11:47:12.046610 1784 consensus_queue.cc:237] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:12.047379 1779 sys_catalog.cc:564] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:12.050576 1785 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } } }
I20260810 11:47:12.050766 1786 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b799cd9aa9be424fa4d856cf0a1f16ae. Latest consensus state: current_term: 1 leader_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } } }
I20260810 11:47:12.051236 1786 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:12.051469 1785 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:12.053212 1795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:12.059435 1795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:12.060286 1779 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:47:12.081992 1795 catalog_manager.cc:1347] Generated new cluster ID: ac8006eee7e147dab9f728396a589e71
I20260810 11:47:12.082198 1795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:12.099819 1795 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:47:12.102113 1795 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:12.115964 1795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Generated new TSK 0
I20260810 11:47:12.117336 1795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:47:12.898041 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:13 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 11:47:13.356083 1828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:13.356629 1828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:13.356858 1828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:13.368854 1828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:13.369027 1828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:13.369282 1828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:13.385859 1828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:13.387981 1828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:13.389990 1828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:13.407950 1838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:13.409011 1840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:13.422484 1842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:13.423702 1828 server_base.cc:1034] running on GCE node
I20260810 11:47:13.424515 1828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:13.425803 1828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:13.427078 1828 hybrid_clock.cc:648] HybridClock initialized: now 1786362433427011 us; error 75 us; skew 500 ppm
I20260810 11:47:13.427438 1828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:13 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:13.430968 1828 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:13.432960 1828 webserver.cc:460] Webserver started at http://127.20.93.129:45973/ using document root <none> and password file <none>
I20260810 11:47:13.433895 1828 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:13.434055 1828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:13.434489 1828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:47:13 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:13.455051 1828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "c0e20251b2ec48b9b91277932aea9842"
format_stamp: "Formatted at 2026-08-10 11:47:13 on dist-test-slave-njxd"
server_key: "f0404d1f92cf6c3c67a2235fb3ffc070"
server_key_iv: "7fbb53f7ff2796b31b68a659337cbcd4"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:13.459074 1828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "c0e20251b2ec48b9b91277932aea9842"
format_stamp: "Formatted at 2026-08-10 11:47:13 on dist-test-slave-njxd"
server_key: "f0404d1f92cf6c3c67a2235fb3ffc070"
server_key_iv: "7fbb53f7ff2796b31b68a659337cbcd4"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:13.470498 1828 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.004s
I20260810 11:47:13.493770 1850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:13.495728 1828 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 11:47:13.495962 1828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "c0e20251b2ec48b9b91277932aea9842"
format_stamp: "Formatted at 2026-08-10 11:47:13 on dist-test-slave-njxd"
server_key: "f0404d1f92cf6c3c67a2235fb3ffc070"
server_key_iv: "7fbb53f7ff2796b31b68a659337cbcd4"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:13.496227 1828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:13.539490 1828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:13.545750 1828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:13.546154 1828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:13.549314 1828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:13.551551 1828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:13.551673 1828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:13.551877 1828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:13.551949 1828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:13.656157 1828 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:44775
I20260810 11:47:13.656178 1972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:44775 every 8 connection(s)
I20260810 11:47:13.658347 1828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:13.661027 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 1828
I20260810 11:47:13.661259 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:47:13.662318 20854 external_mini_cluster.cc:1428] Setting key da6a6735b8e546164d88097599d5ea5a
Aug 10 11:47:13 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:13.699689 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:13.665132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:36713 (local address 127.20.93.190:35501)
0810 11:47:13.678045 (+ 12913us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:13.678054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:13.678082 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:47:13.678162 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:13.678168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:13.678285 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:13.678394 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:13.680791 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:13.682025 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:13.684515 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:13.685219 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:13.685372 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:13.691447 (+ 6075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:13.691455 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:13.691460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:13.691506 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:13.695476 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:13.696554 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:13.696560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:13.696565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:13.696746 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:13.697338 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:13.697343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:13.697346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:13.697777 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:13.697915 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:13.699035 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0810 11:47:13.699287 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12751,"thread_start_us":12607,"threads_started":1}
I20260810 11:47:13.702458 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:13.664069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.129:36713)
0810 11:47:13.674345 (+ 10276us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:13.674414 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:13.677149 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:13.678908 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:13.678922 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:13.679628 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:13.680551 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:13.680579 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:13.682258 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:13.682264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:13.684326 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:13.684346 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:13.685758 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:13.685767 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:13.686067 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:13.686917 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:13.686951 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:13.691117 (+ 4166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:13.695684 (+ 4567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:13.695696 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:13.695717 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:13.696395 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:13.696914 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:13.696919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:13.696922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:13.697196 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:13.698048 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:13.698057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:13.698548 (+ 491us) client_negotiation.cc:769] Sending connection context
0810 11:47:13.700292 (+ 1744us) client_negotiation.cc:240] Negotiation successful
0810 11:47:13.700923 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9845,"thread_start_us":11125,"threads_started":1}
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
I20260810 11:47:13.705629 1974 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:13.706040 1974 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:13.707219 1974 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
I20260810 11:47:13.710168 1740 ts_manager.cc:194] Registered new tserver with Master: c0e20251b2ec48b9b91277932aea9842 (127.20.93.129:44775)
I20260810 11:47:13.712495 1740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:36713
Aug 10 11:47:13 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:47:13.755570 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:14 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362433, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
W20260810 11:47:14.236419 1981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:14.236999 1981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:14.237191 1981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:14.246701 1981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:14.246873 1981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:14.247162 1981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:14.261924 1981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:14.264053 1981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:14.265784 1981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:14.298189 1990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:14.299214 1991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:14.301349 1994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:14.306972 1981 server_base.cc:1034] running on GCE node
I20260810 11:47:14.308051 1981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:14.309321 1981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:14.310560 1981 hybrid_clock.cc:648] HybridClock initialized: now 1786362434310463 us; error 93 us; skew 500 ppm
I20260810 11:47:14.310956 1981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:14 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362434, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:14.314625 1981 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:14.318562 1981 webserver.cc:460] Webserver started at http://127.20.93.130:42507/ using document root <none> and password file <none>
I20260810 11:47:14.319402 1981 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:14.319562 1981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:14.319918 1981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:47:14 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362434, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:14.336570 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "14d942e308d44f6c967f93d085f65b4c"
format_stamp: "Formatted at 2026-08-10 11:47:14 on dist-test-slave-njxd"
server_key: "c3b1946abb0a1b715c2c15bab132350b"
server_key_iv: "b4f1eb5bd99bdeb393080e34bfcbdd70"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:14.337981 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "14d942e308d44f6c967f93d085f65b4c"
format_stamp: "Formatted at 2026-08-10 11:47:14 on dist-test-slave-njxd"
server_key: "c3b1946abb0a1b715c2c15bab132350b"
server_key_iv: "b4f1eb5bd99bdeb393080e34bfcbdd70"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:14.361919 1981 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.015s sys 0.005s
I20260810 11:47:14.392093 2004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:14.394315 1981 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.008s sys 0.001s
I20260810 11:47:14.394634 1981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "14d942e308d44f6c967f93d085f65b4c"
format_stamp: "Formatted at 2026-08-10 11:47:14 on dist-test-slave-njxd"
server_key: "c3b1946abb0a1b715c2c15bab132350b"
server_key_iv: "b4f1eb5bd99bdeb393080e34bfcbdd70"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:14.394997 1981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:14.427852 1981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:14.432614 1981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:14.432976 1981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:14.434239 1981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:14.435810 1981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:14.435890 1981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:14.436067 1981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:14.436123 1981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:14.482314 1981 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:37261
I20260810 11:47:14.482419 2131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:37261 every 8 connection(s)
I20260810 11:47:14.483956 1981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:14.485046 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 1981
I20260810 11:47:14.485296 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:47:14.486083 20854 external_mini_cluster.cc:1428] Setting key e99bbe409120315b76063f909b181f21
Aug 10 11:47:14 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362434, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:14.521970 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:14.489720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:48103 (local address 127.20.93.190:35501)
0810 11:47:14.503999 (+ 14279us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:14.504007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:14.504045 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:47:14.504125 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:14.504132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:14.504245 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:14.504410 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:14.506973 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:14.508127 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:14.510181 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:14.510707 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:14.510872 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:14.514890 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:14.514896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:14.514899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:14.514942 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:14.518119 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:14.519100 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:14.519104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:14.519107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:14.519266 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:14.519840 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:14.519844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:14.519862 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:14.520206 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:14.520391 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:14.521247 (+ 856us) server_negotiation.cc:299] Negotiation successful
0810 11:47:14.521530 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14109,"thread_start_us":14836,"threads_started":1}
I20260810 11:47:14.523499 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:14.487956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.130:48103)
0810 11:47:14.494291 (+ 6335us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:14.494354 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:14.497951 (+ 3597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:14.504838 (+ 6887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:14.504858 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:14.505717 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:14.506681 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:14.506715 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:14.508391 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:14.508397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:14.509970 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:14.509987 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:14.511008 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:14.511016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:14.511280 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:14.512081 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:14.512107 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:14.514580 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:14.518323 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:14.518334 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:14.518355 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:14.518935 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:14.519399 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:14.519404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:14.519424 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:14.519663 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:14.520367 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:14.520377 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:14.520790 (+ 413us) client_negotiation.cc:769] Sending connection context
0810 11:47:14.521117 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 11:47:14.521841 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5977,"thread_start_us":5704,"threads_started":1}
I20260810 11:47:14.525297 2132 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:14.525741 2132 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:14.526599 2132 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
I20260810 11:47:14.528416 1740 ts_manager.cc:194] Registered new tserver with Master: 14d942e308d44f6c967f93d085f65b4c (127.20.93.130:37261)
I20260810 11:47:14.529520 1740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:48103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:47:14.573297 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:14.717020 1974 heartbeater.cc:503] Master 127.20.93.190:35501 was elected leader, sending a full tablet report...
W20260810 11:47:14.969141 2140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:14.969518 2140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:14.969724 2140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:14.977524 2140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:14.977730 2140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:14.977926 2140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:14.988629 2140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:14.990195 2140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:14.991724 2140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:15.008625 2148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:15.008596 2145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:15.008935 2146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:15.010278 2140 server_base.cc:1034] running on GCE node
I20260810 11:47:15.011085 2140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:15.012427 2140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:15.013696 2140 hybrid_clock.cc:648] HybridClock initialized: now 1786362435013625 us; error 68 us; skew 500 ppm
I20260810 11:47:15.014109 2140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362435, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:15.017783 2140 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:15.019537 2140 webserver.cc:460] Webserver started at http://127.20.93.131:41465/ using document root <none> and password file <none>
I20260810 11:47:15.020323 2140 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:15.020471 2140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:15.020826 2140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362435, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:15.040323 2140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "2c2e37ea6b5f49c9886552d59245f307"
format_stamp: "Formatted at 2026-08-10 11:47:15 on dist-test-slave-njxd"
server_key: "f173d077133dbe4fcde6127d67f02f21"
server_key_iv: "1e6033fadbd892fe802fe2cce3be872d"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:15.041522 2140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "2c2e37ea6b5f49c9886552d59245f307"
format_stamp: "Formatted at 2026-08-10 11:47:15 on dist-test-slave-njxd"
server_key: "f173d077133dbe4fcde6127d67f02f21"
server_key_iv: "1e6033fadbd892fe802fe2cce3be872d"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:15.049522 2140 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260810 11:47:15.066787 2156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:15.068828 2140 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:47:15.069160 2140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "2c2e37ea6b5f49c9886552d59245f307"
format_stamp: "Formatted at 2026-08-10 11:47:15 on dist-test-slave-njxd"
server_key: "f173d077133dbe4fcde6127d67f02f21"
server_key_iv: "1e6033fadbd892fe802fe2cce3be872d"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:15.069496 2140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:15.093658 2140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:15.098676 2140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:15.098956 2140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:15.100050 2140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:15.101734 2140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:15.101814 2140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:15.101913 2140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:15.101956 2140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:15.137861 2140 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:33281
I20260810 11:47:15.137903 2276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:33281 every 8 connection(s)
I20260810 11:47:15.139278 2140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:15.140988 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2140
I20260810 11:47:15.141194 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:47:15.141944 20854 external_mini_cluster.cc:1428] Setting key db59fa5d39179465e7cc38574dda050b
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362435, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:15.161988 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:15.142189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:60819 (local address 127.20.93.190:35501)
0810 11:47:15.142934 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:15.142941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:15.144073 (+ 1132us) server_negotiation.cc:407] Connection header received
0810 11:47:15.146104 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:15.146110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:15.146220 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:15.146346 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:15.147897 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.148744 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.150435 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.150908 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.151085 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:15.154963 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:15.154969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:15.154972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:15.155010 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:15.158526 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.159347 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.159352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.159358 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.159518 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.160007 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.160012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.160015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.160320 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:15.160500 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:15.161308 (+ 808us) server_negotiation.cc:299] Negotiation successful
0810 11:47:15.161561 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":427,"threads_started":1}
I20260810 11:47:15.163105 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:15.142934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.131:60819)
0810 11:47:15.143899 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:15.143949 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:15.145838 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:15.146602 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:15.146610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:15.147094 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:15.147722 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:15.147741 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.148911 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.148921 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:15.150284 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:15.150296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.151206 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.151215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:15.151460 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:15.152178 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:15.152201 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:15.154617 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:15.158722 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:15.158731 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:15.158746 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:15.159204 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:15.159637 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:15.159642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:15.159646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:15.159880 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:15.160478 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:15.160487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:15.160914 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 11:47:15.161194 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 11:47:15.161815 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":447,"threads_started":1}
I20260810 11:47:15.164692 2278 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:15.165055 2278 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:15.165910 2278 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
I20260810 11:47:15.168159 1740 ts_manager.cc:194] Registered new tserver with Master: 2c2e37ea6b5f49c9886552d59245f307 (127.20.93.131:33281)
I20260810 11:47:15.169139 1740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:60819
I20260810 11:47:15.175122 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:47:15.177030 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 1828
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:47:15.196918 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:15.178076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.20.93.129:44775)
0810 11:47:15.180291 (+ 2215us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:15.180299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:15.180326 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:47:15.180389 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:15.180395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:15.180618 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:15.180765 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:15.182521 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.183638 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.185148 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.185852 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.185961 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:15.190798 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:15.190816 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:15.190832 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:15.190875 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:15.193776 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.194614 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.194623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.194625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.194810 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.195290 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.195294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.195296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.195694 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:15.195955 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:15.196224 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 11:47:15.196475 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2058,"thread_start_us":1906,"threads_started":1}
I20260810 11:47:15.198434 1967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:15.284548 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 1828
I20260810 11:47:15.305286 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 1981
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:47:15.322607 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:15.306408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.20.93.130:37261)
0810 11:47:15.306942 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:15.306950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:15.306975 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:47:15.307034 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:15.307041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:15.307256 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:15.307390 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:15.308653 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.309762 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.311101 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.311649 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.311747 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:15.316272 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:15.316287 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:15.316303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:15.316349 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:15.319670 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.320460 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.320468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.320470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.320622 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.321027 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.321031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.321034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.321442 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:15.321805 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:15.322056 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 11:47:15.322282 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":241,"threads_started":1}
I20260810 11:47:15.324005 2127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:15.408489 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 1981
I20260810 11:47:15.433270 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2140
Aug 10 11:47:15 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:47:15.452601 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:15.434605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.20.93.131:33281)
0810 11:47:15.435310 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:15.435318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:15.435347 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:47:15.435411 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:15.435418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:15.435655 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:15.435800 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:15.437276 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.438489 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.439955 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:15.440606 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:15.440725 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:15.445654 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:15.445680 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:15.445698 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:15.445740 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:15.448664 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.449524 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.449532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.449535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.449982 (+ 447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:15.450549 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:15.450553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:15.450556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:15.450995 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:15.451303 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:15.451830 (+ 527us) server_negotiation.cc:299] Negotiation successful
0810 11:47:15.452184 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":345,"threads_started":1}
I20260810 11:47:15.454381 2272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:15.542137 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2140
I20260810 11:47:15.562434 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 1708
I20260810 11:47:15.563812 1774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:15.678701 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 1708
I20260810 11:47:15.707897 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35501
--webserver_interface=127.20.93.190
--webserver_port=41711
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35501
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:16.066689 2291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:16.067010 2291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:16.067109 2291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:16.074796 2291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:16.074937 2291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:16.074987 2291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:16.075039 2291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:16.075088 2291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:16.085634 2291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35501
--ranger_config_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=41711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:16.087021 2291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:16.088404 2291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:16.101675 2301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:16.101608 2297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:16.101635 2299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:16.102491 2291 server_base.cc:1034] running on GCE node
I20260810 11:47:16.103344 2291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:16.104863 2291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:16.106094 2291 hybrid_clock.cc:648] HybridClock initialized: now 1786362436106039 us; error 51 us; skew 500 ppm
I20260810 11:47:16.106513 2291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:16 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362436, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:16.114940 2291 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:16.116542 2291 webserver.cc:460] Webserver started at http://127.20.93.190:41711/ using document root <none> and password file <none>
I20260810 11:47:16.117333 2291 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:16.117487 2291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:47:16 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362436, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:16.167044 2291 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 11:47:16.170459 2310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:16.171949 2291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:47:16.172163 2291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "b799cd9aa9be424fa4d856cf0a1f16ae"
format_stamp: "Formatted at 2026-08-10 11:47:11 on dist-test-slave-njxd"
server_key: "7303bb1a7b15edb5e42aa2bea92e9b4f"
server_key_iv: "8ed44997bc90e59d8acbf874d19740ba"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:16.173195 2291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:16.214131 2291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:16.218608 2291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:16.218951 2291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:16.238152 2291 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35501
I20260810 11:47:16.238171 2366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35501 every 8 connection(s)
I20260810 11:47:16.239732 2291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:16.246905 2367 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:47:16.250041 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2291
I20260810 11:47:16.250418 2367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Bootstrap starting.
I20260810 11:47:16.250978 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:44775
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:35501
--webserver_port=45973
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:16.270279 2367 log.cc:824] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:16.280939 2367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:47:16.281613 2367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Bootstrap complete.
I20260810 11:47:16.292155 2367 raft_consensus.cc:348] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:16.293149 2367 raft_consensus.cc:729] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b799cd9aa9be424fa4d856cf0a1f16ae, State: Initialized, Role: FOLLOWER
I20260810 11:47:16.293901 2367 consensus_queue.cc:260] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:16.294178 2367 raft_consensus.cc:388] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:16.294329 2367 raft_consensus.cc:482] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:16.294552 2367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:47:16.298959 2367 raft_consensus.cc:504] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:16.299501 2367 leader_election.cc:302] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b799cd9aa9be424fa4d856cf0a1f16ae; no voters:
I20260810 11:47:16.300275 2367 leader_election.cc:288] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:47:16.300428 2373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:47:16.301769 2373 raft_consensus.cc:686] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [term 2 LEADER]: Becoming Leader. State: Replica: b799cd9aa9be424fa4d856cf0a1f16ae, State: Running, Role: LEADER
I20260810 11:47:16.302469 2373 consensus_queue.cc:237] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } }
I20260810 11:47:16.302852 2367 sys_catalog.cc:564] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:16.305356 2375 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b799cd9aa9be424fa4d856cf0a1f16ae. Latest consensus state: current_term: 2 leader_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } } }
I20260810 11:47:16.305768 2375 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:16.305449 2374 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b799cd9aa9be424fa4d856cf0a1f16ae" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35501 } } }
I20260810 11:47:16.305954 2374 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:16.311589 2379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:16.321658 2379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:16.322662 2367 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:47:16.323411 2379 catalog_manager.cc:1259] Loaded cluster ID: ac8006eee7e147dab9f728396a589e71
I20260810 11:47:16.323621 2379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:16.328156 2379 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:16.330076 2379 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Loaded TSK: 0
I20260810 11:47:16.341703 2379 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:47:16.746048 2369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:16.746598 2369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:16.746811 2369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:16.759191 2369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:16.759440 2369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:16.759711 2369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:16.776840 2369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:44775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=45973
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:16.779011 2369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:16.781013 2369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:16.816954 2407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:16.816828 2408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:16.817776 2410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:16.819890 2369 server_base.cc:1034] running on GCE node
I20260810 11:47:16.820683 2369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:16.822022 2369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:16.823323 2369 hybrid_clock.cc:648] HybridClock initialized: now 1786362436823251 us; error 56 us; skew 500 ppm
I20260810 11:47:16.823776 2369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:16 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362436, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:16.828440 2369 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:16.830889 2369 webserver.cc:460] Webserver started at http://127.20.93.129:45973/ using document root <none> and password file <none>
I20260810 11:47:16.831902 2369 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:16.832064 2369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:47:16 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362436, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:16.889225 2369 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.003s sys 0.012s
I20260810 11:47:16.897490 2418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:16.899621 2369 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.004s
I20260810 11:47:16.899817 2369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "c0e20251b2ec48b9b91277932aea9842"
format_stamp: "Formatted at 2026-08-10 11:47:13 on dist-test-slave-njxd"
server_key: "f0404d1f92cf6c3c67a2235fb3ffc070"
server_key_iv: "7fbb53f7ff2796b31b68a659337cbcd4"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:16.901400 2369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:16.919570 2369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:16.924181 2369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:16.924495 2369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:16.925781 2369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:16.927587 2369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:16.927755 2369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:47:16.927927 2369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:16.928071 2369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:17.013901 2369 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:44775
I20260810 11:47:17.013952 2541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:44775 every 8 connection(s)
I20260810 11:47:17.015538 2369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:17.022527 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2369
I20260810 11:47:17.023433 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:37261
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:35501
--webserver_port=42507
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:17 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362436, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:17.061127 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:17.019950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.129:56913)
0810 11:47:17.023992 (+ 4042us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:17.024049 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:17.026931 (+ 2882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:17.036585 (+ 9654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:17.036600 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:17.037376 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:17.038424 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:17.038459 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.041133 (+ 2674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.041141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:17.043320 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:17.043340 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.045426 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.045437 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:17.045836 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:17.046721 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:17.046763 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:17.049834 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:17.055034 (+ 5200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:17.055045 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:17.055064 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:17.055779 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:17.056457 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:17.056463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:17.056467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:17.056762 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:17.058038 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:17.058048 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:17.058542 (+ 494us) client_negotiation.cc:769] Sending connection context
0810 11:47:17.058892 (+ 350us) client_negotiation.cc:240] Negotiation successful
0810 11:47:17.059554 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3685,"thread_start_us":9752,"threads_started":1}
I20260810 11:47:17.067006 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:17.019732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:56913 (local address 127.20.93.190:35501)
0810 11:47:17.033478 (+ 13746us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:17.033504 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:17.033845 (+ 341us) server_negotiation.cc:407] Connection header received
0810 11:47:17.034426 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:17.034453 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:17.035119 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:17.035688 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:17.038768 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.040850 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.043558 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.044734 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.044942 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:17.050227 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:17.050239 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:17.050264 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:17.050324 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:17.054786 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:17.056002 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:17.056013 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:17.056023 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:17.056289 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:17.056917 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:17.056922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:17.056925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:17.057747 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:17.057956 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:17.064979 (+ 7023us) server_negotiation.cc:299] Negotiation successful
0810 11:47:17.065408 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13383,"thread_start_us":13707,"threads_started":1}
I20260810 11:47:17.071720 2542 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:17.072248 2542 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:17.075012 2542 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
W20260810 11:47:17.080909 2542 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:35501 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:47:17.081346 2542 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:17.081813 2542 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
I20260810 11:47:17.083249 2542 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:17.083563 2542 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
W20260810 11:47:17.084789 2542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:47:17.544270 2546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:17.544749 2546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:17.544946 2546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:17.558673 2546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:17.558900 2546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:17.559134 2546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:17.577965 2546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:37261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=42507
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:17.580049 2546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:17.582154 2546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:17.605989 2561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:17.607152 2562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:17.609498 2565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:17.614136 2546 server_base.cc:1034] running on GCE node
I20260810 11:47:17.615217 2546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:17.633968 2546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:17.635304 2546 hybrid_clock.cc:648] HybridClock initialized: now 1786362437635198 us; error 103 us; skew 500 ppm
I20260810 11:47:17.635864 2546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:17 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:17.640873 2546 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:17.643033 2546 webserver.cc:460] Webserver started at http://127.20.93.130:42507/ using document root <none> and password file <none>
I20260810 11:47:17.643956 2546 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:17.644155 2546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:47:17 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:17.699344 2546 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.004s
I20260810 11:47:17.712443 2576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:17.715070 2546 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.008s
I20260810 11:47:17.715337 2546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "14d942e308d44f6c967f93d085f65b4c"
format_stamp: "Formatted at 2026-08-10 11:47:14 on dist-test-slave-njxd"
server_key: "c3b1946abb0a1b715c2c15bab132350b"
server_key_iv: "b4f1eb5bd99bdeb393080e34bfcbdd70"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:17.717098 2546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 11:47:17 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:17.747628 2546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:17.753727 2546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:17.754072 2546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:17.755456 2546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:17.759266 2546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:17.759397 2546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:17.759518 2546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:17.759595 2546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:17.815660 2546 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:37261
I20260810 11:47:17.815718 2698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:37261 every 8 connection(s)
I20260810 11:47:17.817790 2546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:17.819231 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2546
I20260810 11:47:17.820079 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.93.148:50457/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:33281
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:35501
--webserver_port=41465
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:17 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:17.870728 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:17.823148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:53683 (local address 127.20.93.190:35501)
0810 11:47:17.844654 (+ 21506us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:17.844667 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:17.850199 (+ 5532us) server_negotiation.cc:407] Connection header received
0810 11:47:17.850288 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:17.850293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:17.850371 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:17.850513 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:17.852812 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.854079 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.856802 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.857459 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.857597 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:17.862588 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:17.862596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:17.862601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:17.862648 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:17.866418 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:17.867518 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:17.867524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:17.867528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:17.867695 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:17.868337 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:17.868343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:17.868346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:17.868735 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:17.868878 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:17.870042 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0810 11:47:17.870320 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21283,"thread_start_us":21550,"threads_started":1}
I20260810 11:47:17.873710 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:17.822359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.130:53683)
0810 11:47:17.845725 (+ 23366us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:17.845832 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:17.849609 (+ 3777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:17.850843 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:17.850857 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:17.851606 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:17.852531 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:17.852560 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.854339 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.854345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:17.856579 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:17.856597 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:17.858023 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:17.858031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:17.858360 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:17.859305 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:17.859339 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:17.862258 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:17.866637 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:17.866648 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:17.866670 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:17.867346 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:17.867877 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:17.867882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:17.867886 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:17.868186 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:17.868996 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:17.869004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:17.869508 (+ 504us) client_negotiation.cc:769] Sending connection context
0810 11:47:17.871362 (+ 1854us) client_negotiation.cc:240] Negotiation successful
0810 11:47:17.872050 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22822,"thread_start_us":26645,"threads_started":1}
I20260810 11:47:17.876010 2701 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:17.876371 2701 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:17.877425 2701 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
I20260810 11:47:17.879827 2327 ts_manager.cc:194] Registered new tserver with Master: 14d942e308d44f6c967f93d085f65b4c (127.20.93.130:37261)
I20260810 11:47:17.882253 2327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:53683
I20260810 11:47:18.085024 2542 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:18.085556 2542 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
I20260810 11:47:18.087050 2327 ts_manager.cc:194] Registered new tserver with Master: c0e20251b2ec48b9b91277932aea9842 (127.20.93.129:44775)
I20260810 11:47:18.088261 2327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:56913
Aug 10 11:47:18 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
W20260810 11:47:18.394855 2705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:18.395236 2705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:18.395397 2705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:18.403287 2705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:18.403456 2705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:18.403637 2705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:18.414888 2705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.93.148:50457/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:33281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=41465
--tserver_master_addrs=127.20.93.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:18.416815 2705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:18.418807 2705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:18.436210 2710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:18.437808 2711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:18.446135 2705 server_base.cc:1034] running on GCE node
W20260810 11:47:18.446453 2714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:18.447439 2705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:18.449240 2705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:18.450680 2705 hybrid_clock.cc:648] HybridClock initialized: now 1786362438450477 us; error 192 us; skew 500 ppm
I20260810 11:47:18.451231 2705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:18 dist-test-slave-njxd krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362438, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:18.456053 2705 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:18.458485 2705 webserver.cc:460] Webserver started at http://127.20.93.131:41465/ using document root <none> and password file <none>
I20260810 11:47:18.459700 2705 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:18.459931 2705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:47:18 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362438, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:18.492045 2705 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.003s
I20260810 11:47:18.496515 2723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:18.498668 2705 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:47:18.498925 2705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "2c2e37ea6b5f49c9886552d59245f307"
format_stamp: "Formatted at 2026-08-10 11:47:15 on dist-test-slave-njxd"
server_key: "f173d077133dbe4fcde6127d67f02f21"
server_key_iv: "1e6033fadbd892fe802fe2cce3be872d"
server_key_version: "kuduclusterkey@0"
I20260810 11:47:18.500525 2705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:18.529482 2705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:18.535404 2705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:18.535751 2705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:18.537318 2705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:18.539800 2705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:18.539935 2705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:18.540047 2705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:18.540120 2705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:47:18 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362437, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for HTTP/127.20.93.148@KRBTEST.COM
I20260810 11:47:18.638489 2705 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:33281
I20260810 11:47:18.638542 2852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:33281 every 8 connection(s)
I20260810 11:47:18.640470 2705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:18.646622 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2705
I20260810 11:47:18.706512 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.648386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.20.93.190:35501)
0810 11:47:18.677985 (+ 29599us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.677993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.678018 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:47:18.678095 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.678101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.678207 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.678364 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:18.687959 (+ 9595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.689119 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.692418 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.693040 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.693149 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.697262 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:18.697268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:18.697272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:18.697326 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:18.701295 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.703073 (+ 1778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.703080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.703084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.703261 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.703823 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.703827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.703830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.704253 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:18.704581 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.705844 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.706118 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29459,"thread_start_us":9798,"threads_started":1}
Aug 10 11:47:18 dist-test-slave-njxd krb5kdc[1223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362438, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:18.720000 2860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.645900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35927 (local address 127.20.93.190:35501)
0810 11:47:18.659221 (+ 13321us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.659232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.679607 (+ 20375us) server_negotiation.cc:407] Connection header received
0810 11:47:18.684045 (+ 4438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.684054 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.684195 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.684350 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:18.698522 (+ 14172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.700219 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.703808 (+ 3589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.704562 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.704657 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.710997 (+ 6340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:18.711005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:18.711010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:18.711065 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:18.714724 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.715813 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.715821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.715828 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.716002 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.716730 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.716735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.716739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.717108 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:18.717277 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.719293 (+ 2016us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.719575 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12493,"thread_start_us":39223,"threads_started":1}
I20260810 11:47:18.721365 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.647692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35501 (local address 127.20.93.131:35927)
0810 11:47:18.679409 (+ 31717us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:18.679479 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:18.683216 (+ 3737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:18.688055 (+ 4839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:18.688073 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:18.688783 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:18.697888 (+ 9105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.697928 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.700467 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.700474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.703610 (+ 3136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.703641 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.705047 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.705056 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.705400 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.706388 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:18.706421 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:18.710669 (+ 4248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:18.714939 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:18.714950 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:18.714971 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:18.715647 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:18.716295 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:18.716299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:18.716301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:18.716552 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:18.717435 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:18.717445 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:18.717997 (+ 552us) client_negotiation.cc:769] Sending connection context
0810 11:47:18.719142 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0810 11:47:18.719846 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28540,"thread_start_us":28610,"threads_started":1}
I20260810 11:47:18.723207 2853 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35501
I20260810 11:47:18.723572 2853 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:18.724577 2853 heartbeater.cc:511] Master 127.20.93.190:35501 requested a full tablet report, sending...
I20260810 11:47:18.726457 2327 ts_manager.cc:194] Registered new tserver with Master: 2c2e37ea6b5f49c9886552d59245f307 (127.20.93.131:33281)
I20260810 11:47:18.727635 2327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:35927
I20260810 11:47:18.739318 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:47:18.778769 2860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.747791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.20.93.190:35501)
0810 11:47:18.748001 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.748006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.759885 (+ 11879us) server_negotiation.cc:407] Connection header received
0810 11:47:18.760512 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.760520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.760617 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.760696 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:18.763226 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.764345 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.766506 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.767178 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.767334 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.771040 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:18.771048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:18.771054 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:18.771104 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:18.774964 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.776039 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.776046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.776050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.776249 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:18.776759 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:18.776764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:18.776768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:18.777171 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:18.777415 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.778112 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.778403 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:47:18.786747 2327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:47:18.798954 2327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:47:18.878664 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.841996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.20.93.130:37261)
0810 11:47:18.846077 (+ 4081us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.846088 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.856210 (+ 10122us) server_negotiation.cc:407] Connection header received
0810 11:47:18.856347 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.856359 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.856684 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.856915 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:18.867285 (+ 10370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.869590 (+ 2305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.874798 (+ 5208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.876655 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.876797 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.877083 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.878045 (+ 962us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.878257 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3899,"thread_start_us":3755,"threads_started":1}
I20260810 11:47:18.883476 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.846923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44775 (local address 127.0.0.1:55976)
0810 11:47:18.871284 (+ 24361us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:18.871307 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:18.871453 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:18.873020 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:18.873029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:18.873082 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:18.873679 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.873714 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.876610 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.876620 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.879299 (+ 2679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.879320 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.882346 (+ 3026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.882356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.882639 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.882697 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 11:47:18.882811 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 11:47:18.883033 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24195,"thread_start_us":24020,"threads_started":1}
I20260810 11:47:18.883080 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.841236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:37261 (local address 127.0.0.1:43674)
0810 11:47:18.854848 (+ 13612us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:18.854904 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:18.855152 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:18.865760 (+ 10608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:18.865772 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:18.865821 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:18.866384 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.866405 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.869772 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.869779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.874493 (+ 4714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.874536 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.877348 (+ 2812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.877356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.877687 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.877768 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 11:47:18.879013 (+ 1245us) client_negotiation.cc:240] Negotiation successful
0810 11:47:18.882665 (+ 3652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13214,"thread_start_us":14145,"threads_started":1}
I20260810 11:47:18.884685 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.847663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.20.93.129:44775)
0810 11:47:18.867089 (+ 19426us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.867104 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.871619 (+ 4515us) server_negotiation.cc:407] Connection header received
0810 11:47:18.871909 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.871926 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.872289 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.872594 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:18.874035 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.876339 (+ 2304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.879674 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.881564 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.881805 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.882139 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.884157 (+ 2018us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.884366 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19215,"thread_start_us":19060,"threads_started":1}
I20260810 11:47:18.887552 2701 heartbeater.cc:503] Master 127.20.93.190:35501 was elected leader, sending a full tablet report...
I20260810 11:47:18.888928 2628 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 (DEFAULT_TABLE table=test-table [id=24cc135854b34fe9aac6948f7d988ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:18.890805 2628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2dbe3a3fa64230bd7210970f84f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:18.892554 2474 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 (DEFAULT_TABLE table=test-table [id=24cc135854b34fe9aac6948f7d988ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:18.896159 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.877918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:33281 (local address 127.0.0.1:53498)
0810 11:47:18.883780 (+ 5862us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:18.883801 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:18.883937 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:18.886431 (+ 2494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:18.886436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:18.886476 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:18.886695 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.886710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.889082 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.889088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.891060 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:18.891083 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.895180 (+ 4097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.895188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:18.895457 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.895501 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 11:47:18.895616 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:47:18.895799 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5738,"thread_start_us":10005,"threads_started":1}
I20260810 11:47:18.897233 2474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2dbe3a3fa64230bd7210970f84f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:18.902626 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:18.872068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.20.93.131:33281)
0810 11:47:18.883677 (+ 11609us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:18.883689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:18.885390 (+ 1701us) server_negotiation.cc:407] Connection header received
0810 11:47:18.885520 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:18.885529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:18.885861 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:18.886075 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:18.887237 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.888896 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.892599 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:18.894273 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:18.894423 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:18.894707 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:18.901957 (+ 7250us) server_negotiation.cc:299] Negotiation successful
0810 11:47:18.902200 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11372,"thread_start_us":11179,"threads_started":1}
I20260810 11:47:18.907518 2779 tablet_service.cc:1467] Processing CreateTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 (DEFAULT_TABLE table=test-table [id=24cc135854b34fe9aac6948f7d988ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:18.909435 2779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d2dbe3a3fa64230bd7210970f84f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:18.927526 2878 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Bootstrap starting.
I20260810 11:47:18.931677 2878 tablet_bootstrap.cc:654] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:18.933485 2878 log.cc:824] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:18.938308 2880 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Bootstrap starting.
I20260810 11:47:18.946525 2879 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Bootstrap starting.
I20260810 11:47:18.958983 2879 tablet_bootstrap.cc:654] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:18.961403 2880 tablet_bootstrap.cc:654] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:18.963181 2880 log.cc:824] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:18.963891 2878 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: No bootstrap required, opened a new log
I20260810 11:47:18.964239 2878 ts_tablet_manager.cc:1397] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Time spent bootstrapping tablet: real 0.041s user 0.023s sys 0.009s
I20260810 11:47:18.965796 2879 log.cc:824] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:18.974617 2879 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: No bootstrap required, opened a new log
I20260810 11:47:18.975068 2879 ts_tablet_manager.cc:1397] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.004s
I20260810 11:47:18.975531 2878 raft_consensus.cc:348] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.976053 2878 raft_consensus.cc:374] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:18.976166 2878 raft_consensus.cc:729] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e20251b2ec48b9b91277932aea9842, State: Initialized, Role: FOLLOWER
I20260810 11:47:18.976231 2880 tablet_bootstrap.cc:492] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: No bootstrap required, opened a new log
I20260810 11:47:18.976599 2880 ts_tablet_manager.cc:1397] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.004s
I20260810 11:47:18.976948 2878 consensus_queue.cc:260] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.978339 2542 heartbeater.cc:503] Master 127.20.93.190:35501 was elected leader, sending a full tablet report...
I20260810 11:47:18.979187 2878 ts_tablet_manager.cc:1428] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 11:47:18.986836 2880 raft_consensus.cc:348] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.987404 2880 raft_consensus.cc:374] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:18.987515 2880 raft_consensus.cc:729] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d942e308d44f6c967f93d085f65b4c, State: Initialized, Role: FOLLOWER
I20260810 11:47:18.987253 2879 raft_consensus.cc:348] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.987758 2879 raft_consensus.cc:374] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:18.987859 2879 raft_consensus.cc:729] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e37ea6b5f49c9886552d59245f307, State: Initialized, Role: FOLLOWER
I20260810 11:47:18.988368 2880 consensus_queue.cc:260] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.988510 2879 consensus_queue.cc:260] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:18.990422 2880 ts_tablet_manager.cc:1428] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260810 11:47:18.991485 2853 heartbeater.cc:503] Master 127.20.93.190:35501 was elected leader, sending a full tablet report...
I20260810 11:47:18.992226 2879 ts_tablet_manager.cc:1428] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
W20260810 11:47:19.025007 2543 tablet.cc:2369] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:19.072307 2895 raft_consensus.cc:482] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:19.072700 2895 raft_consensus.cc:504] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:19.075287 2895 leader_election.cc:288] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14d942e308d44f6c967f93d085f65b4c (127.20.93.130:37261), c0e20251b2ec48b9b91277932aea9842 (127.20.93.129:44775)
I20260810 11:47:19.085876 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.076434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:37261 (local address 127.20.93.131:32909)
0810 11:47:19.076682 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:19.076729 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:19.076935 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:19.078361 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:19.078370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:19.078412 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:19.078714 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:19.078730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.080642 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.080651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:19.083109 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:19.083125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.085028 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.085035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:19.085267 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.085313 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 11:47:19.085410 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 11:47:19.085627 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 11:47:19.086553 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.077284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:32909 (local address 127.20.93.130:37261)
0810 11:47:19.077603 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.077610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.077670 (+ 60us) server_negotiation.cc:407] Connection header received
0810 11:47:19.077782 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.077787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.077888 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.078021 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:19.078936 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.080400 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.083397 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.084608 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.084801 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.084878 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.086241 (+ 1363us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.086373 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260810 11:47:19.087533 2650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2dbe3a3fa64230bd7210970f84f884" candidate_uuid: "2c2e37ea6b5f49c9886552d59245f307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d942e308d44f6c967f93d085f65b4c" is_pre_election: true
I20260810 11:47:19.088137 2650 raft_consensus.cc:2454] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2e37ea6b5f49c9886552d59245f307 in term 0.
I20260810 11:47:19.088918 2725 leader_election.cc:302] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d942e308d44f6c967f93d085f65b4c, 2c2e37ea6b5f49c9886552d59245f307; no voters:
I20260810 11:47:19.089449 2895 raft_consensus.cc:2781] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:47:19.089656 2895 raft_consensus.cc:482] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:47:19.089711 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.076246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:47533 (local address 127.20.93.129:44775)
0810 11:47:19.077093 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.077102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.082296 (+ 5194us) server_negotiation.cc:407] Connection header received
0810 11:47:19.082482 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.082488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.082588 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.082690 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:19.083526 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.084627 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.087160 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.088379 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.088519 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.088595 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.089265 (+ 670us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.089427 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621}
I20260810 11:47:19.089804 2895 raft_consensus.cc:3037] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:19.089895 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.077632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44775 (local address 127.20.93.131:47533)
0810 11:47:19.082151 (+ 4519us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:19.082174 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:19.082328 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:19.082731 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:19.082736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:19.082770 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:19.083286 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:19.083300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.084855 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.084861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:19.086945 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:19.086961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.088809 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.088816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:19.089025 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.089064 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:47:19.089217 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 11:47:19.089373 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4371,"thread_start_us":4118,"threads_started":1}
I20260810 11:47:19.091042 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2dbe3a3fa64230bd7210970f84f884" candidate_uuid: "2c2e37ea6b5f49c9886552d59245f307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e20251b2ec48b9b91277932aea9842" is_pre_election: true
I20260810 11:47:19.091655 2494 raft_consensus.cc:2454] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2e37ea6b5f49c9886552d59245f307 in term 0.
I20260810 11:47:19.093364 2895 raft_consensus.cc:504] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:19.094061 2895 leader_election.cc:288] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [CANDIDATE]: Term 1 election: Requested vote from peers 14d942e308d44f6c967f93d085f65b4c (127.20.93.130:37261), c0e20251b2ec48b9b91277932aea9842 (127.20.93.129:44775)
I20260810 11:47:19.094576 2650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2dbe3a3fa64230bd7210970f84f884" candidate_uuid: "2c2e37ea6b5f49c9886552d59245f307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d942e308d44f6c967f93d085f65b4c"
I20260810 11:47:19.094798 2650 raft_consensus.cc:3037] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:19.097129 2650 raft_consensus.cc:2454] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2e37ea6b5f49c9886552d59245f307 in term 1.
I20260810 11:47:19.097599 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d2dbe3a3fa64230bd7210970f84f884" candidate_uuid: "2c2e37ea6b5f49c9886552d59245f307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e20251b2ec48b9b91277932aea9842"
I20260810 11:47:19.097657 2725 leader_election.cc:302] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d942e308d44f6c967f93d085f65b4c, 2c2e37ea6b5f49c9886552d59245f307; no voters:
I20260810 11:47:19.097848 2494 raft_consensus.cc:3037] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:19.097999 2895 raft_consensus.cc:2781] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 1 FOLLOWER]: Leader election won for term 1
W20260810 11:47:19.100554 2702 tablet.cc:2369] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:19.101704 2895 raft_consensus.cc:686] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 1 LEADER]: Becoming Leader. State: Replica: 2c2e37ea6b5f49c9886552d59245f307, State: Running, Role: LEADER
I20260810 11:47:19.102365 2494 raft_consensus.cc:2454] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2e37ea6b5f49c9886552d59245f307 in term 1.
I20260810 11:47:19.102806 2895 consensus_queue.cc:237] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } }
I20260810 11:47:19.108115 2327 catalog_manager.cc:5495] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c2e37ea6b5f49c9886552d59245f307 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "2c2e37ea6b5f49c9886552d59245f307" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2e37ea6b5f49c9886552d59245f307" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 33281 } health_report { overall_health: HEALTHY } } }
I20260810 11:47:19.157974 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.151632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.20.93.131:33281)
0810 11:47:19.151835 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.151842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.151887 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:47:19.152129 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.152134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.152259 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.152423 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:47:19.152801 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.153955 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.155351 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.155923 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.156051 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.156616 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:47:19.157378 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:47:19.157531 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.157616 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.157704 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:47:19.168107 2650 raft_consensus.cc:1261] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 1 FOLLOWER]: Refusing update from remote peer 2c2e37ea6b5f49c9886552d59245f307: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:19.168107 2494 raft_consensus.cc:1261] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 1 FOLLOWER]: Refusing update from remote peer 2c2e37ea6b5f49c9886552d59245f307: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:19.169263 2897 consensus_queue.cc:1035] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e20251b2ec48b9b91277932aea9842" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:19.169667 2895 consensus_queue.cc:1035] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d942e308d44f6c967f93d085f65b4c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:19.187927 2900 mvcc.cc:205] Tried to move back new op lower bound from 7316940550819835904 to 7316940550570131456. Current Snapshot: MvccSnapshot[applied={T|T < 7316940550819835904}]
I20260810 11:47:19.189296 2902 mvcc.cc:205] Tried to move back new op lower bound from 7316940550819835904 to 7316940550570131456. Current Snapshot: MvccSnapshot[applied={T|T < 7316940550819835904}]
I20260810 11:47:19.203649 2327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:19.203989 2327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:19.209856 2327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b799cd9aa9be424fa4d856cf0a1f16ae: Sending DeleteTablet for 3 replicas of tablet 7d2dbe3a3fa64230bd7210970f84f884
I20260810 11:47:19.211323 2628 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:19 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:43674
I20260810 11:47:19.212431 2474 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:19 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:55976
I20260810 11:47:19.214135 2779 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d2dbe3a3fa64230bd7210970f84f884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:19 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:53498
I20260810 11:47:19.215891 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2369
I20260810 11:47:19.225589 2910 tablet_replica.cc:330] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: stopping tablet replica
I20260810 11:47:19.226409 2910 raft_consensus.cc:2229] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:19.227216 2910 raft_consensus.cc:2258] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:19.232309 2910 ts_tablet_manager.cc:1905] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:19.240494 2909 tablet_replica.cc:330] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: stopping tablet replica
I20260810 11:47:19.241110 2909 raft_consensus.cc:2229] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:19.241797 2910 ts_tablet_manager.cc:1918] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:19.242005 2910 log.cc:1196] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/7d2dbe3a3fa64230bd7210970f84f884
I20260810 11:47:19.242725 2910 ts_tablet_manager.cc:1939] T 7d2dbe3a3fa64230bd7210970f84f884 P c0e20251b2ec48b9b91277932aea9842: Deleting consensus metadata
I20260810 11:47:19.244999 2313 catalog_manager.cc:4841] TS c0e20251b2ec48b9b91277932aea9842 (127.20.93.129:44775): tablet 7d2dbe3a3fa64230bd7210970f84f884 (table test-table [id=24cc135854b34fe9aac6948f7d988ccb]) successfully deleted
I20260810 11:47:19.241855 2909 raft_consensus.cc:2258] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:19.248006 2909 ts_tablet_manager.cc:1905] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:19.250860 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.217020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.20.93.129:44775)
0810 11:47:19.217251 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.217260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.228454 (+ 11194us) server_negotiation.cc:407] Connection header received
0810 11:47:19.228760 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.228766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.228861 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.228948 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:19.231286 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.232667 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.235192 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.235878 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.235990 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.242194 (+ 6204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:19.242212 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:19.242234 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:19.242290 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:19.246752 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.247907 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.247917 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.247920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.248099 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.248703 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.248710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.248716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.249206 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:19.249511 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.250221 (+ 710us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.250475 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:47:19.252837 2536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:19.253477 2911 tablet_replica.cc:330] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: stopping tablet replica
I20260810 11:47:19.254016 2911 raft_consensus.cc:2229] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:47:19.254806 2911 raft_consensus.cc:2258] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:19.256299 2909 ts_tablet_manager.cc:1918] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:19.256533 2909 log.cc:1196] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/7d2dbe3a3fa64230bd7210970f84f884
I20260810 11:47:19.257136 2909 ts_tablet_manager.cc:1939] T 7d2dbe3a3fa64230bd7210970f84f884 P 14d942e308d44f6c967f93d085f65b4c: Deleting consensus metadata
I20260810 11:47:19.259117 2312 catalog_manager.cc:4841] TS 14d942e308d44f6c967f93d085f65b4c (127.20.93.130:37261): tablet 7d2dbe3a3fa64230bd7210970f84f884 (table test-table [id=24cc135854b34fe9aac6948f7d988ccb]) successfully deleted
I20260810 11:47:19.261688 2911 ts_tablet_manager.cc:1905] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:19.269845 2911 ts_tablet_manager.cc:1918] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:19.270154 2911 log.cc:1196] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/7d2dbe3a3fa64230bd7210970f84f884
I20260810 11:47:19.270814 2911 ts_tablet_manager.cc:1939] T 7d2dbe3a3fa64230bd7210970f84f884 P 2c2e37ea6b5f49c9886552d59245f307: Deleting consensus metadata
I20260810 11:47:19.272816 2313 catalog_manager.cc:4841] TS 2c2e37ea6b5f49c9886552d59245f307 (127.20.93.131:33281): tablet 7d2dbe3a3fa64230bd7210970f84f884 (table test-table [id=24cc135854b34fe9aac6948f7d988ccb]) successfully deleted
I20260810 11:47:19.421962 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2369
I20260810 11:47:19.442443 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2546
I20260810 11:47:19.458499 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.443478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.20.93.130:37261)
0810 11:47:19.443697 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.443703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.443728 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:47:19.443860 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.443866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.443946 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.444065 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:19.446069 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.446833 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.448182 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.448629 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.448791 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.451765 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:19.451783 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:19.451798 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:19.451840 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:19.455062 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.455940 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.455949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.455951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.456119 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.456556 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.456561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.456565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.456951 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:19.457288 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.457969 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.458210 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:47:19.459371 2693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:19.553092 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2546
I20260810 11:47:19.582185 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2705
I20260810 11:47:19.598455 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:19.583104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.20.93.131:33281)
0810 11:47:19.583318 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:19.583325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:19.583346 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:47:19.583494 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:19.583504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:19.583587 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:19.583716 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:19.585067 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.586021 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.587498 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:19.588097 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:19.588260 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:19.591060 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:19.591076 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:19.591094 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:19.591138 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:19.594456 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.595282 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.595332 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.595336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.595533 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:19.596088 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:19.596093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:19.596096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:19.596658 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:19.596983 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:19.597599 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 11:47:19.597980 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:47:19.599262 2848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:19.728186 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2705
I20260810 11:47:19.752604 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2291
I20260810 11:47:19.753928 2361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:19.869392 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2291
I20260810 11:47:19.899787 20854 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 11:47:20.246309 20854 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 11:47:20.246565 20854 mini_ranger.cc:66] Stopping Ranger...
I20260810 11:47:20.593088 20854 mini_ranger.cc:68] Stopped Ranger
2026-08-10 11:47:20.593 UTC [1256] LOG: received smart shutdown request
2026-08-10 11:47:20.596 UTC [1256] LOG: background worker "logical replication launcher" (PID 1265) exited with exit code 1
2026-08-10 11:47:20.597 UTC [1260] LOG: shutting down
2026-08-10 11:47:20.792 UTC [1256] LOG: database system is shut down
2026-08-10T11:47:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56613 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:47:21 dist-test-slave-njxd krb5kdc[2926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:47:21 dist-test-slave-njxd krb5kdc[2926](info): set up 2 sockets
Aug 10 11:47:21 dist-test-slave-njxd krb5kdc[2926](info): commencing operation
krb5kdc: starting...
W20260810 11:47:22.138094 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:47:22 dist-test-slave-njxd krb5kdc[2926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:47:22.442704 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38645
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:22.808763 2940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:22.809123 2940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:22.809232 2940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:22.816658 2940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:22.816787 2940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:22.816835 2940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:22.816900 2940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:22.816953 2940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:22.827235 2940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:22.828604 2940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:22.829962 2940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:22.842753 2947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:22.842856 2945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:22.842787 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:22.848866 2940 server_base.cc:1034] running on GCE node
I20260810 11:47:22.849818 2940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:22.851279 2940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:22.852514 2940 hybrid_clock.cc:648] HybridClock initialized: now 1786362442852472 us; error 40 us; skew 500 ppm
I20260810 11:47:22.852883 2940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:22 dist-test-slave-njxd krb5kdc[2926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:22.859387 2940 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:22.861128 2940 webserver.cc:460] Webserver started at http://127.20.93.190:34135/ using document root <none> and password file <none>
I20260810 11:47:22.861982 2940 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:22.862138 2940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:22.862468 2940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:22.865564 2940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a4c5c52b301e46198b7a75b2cd501aed"
format_stamp: "Formatted at 2026-08-10 11:47:22 on dist-test-slave-njxd"
server_key: "9790c7b23d43f88d9ef2f97dab3f7ac9"
server_key_iv: "4a39a4c89c1d8a679e238165d85824b1"
server_key_version: "encryptionkey@0"
I20260810 11:47:22.866515 2940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a4c5c52b301e46198b7a75b2cd501aed"
format_stamp: "Formatted at 2026-08-10 11:47:22 on dist-test-slave-njxd"
server_key: "9790c7b23d43f88d9ef2f97dab3f7ac9"
server_key_iv: "4a39a4c89c1d8a679e238165d85824b1"
server_key_version: "encryptionkey@0"
I20260810 11:47:22.872015 2940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:47:22.875828 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:22.877216 2940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:47:22.877415 2940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a4c5c52b301e46198b7a75b2cd501aed"
format_stamp: "Formatted at 2026-08-10 11:47:22 on dist-test-slave-njxd"
server_key: "9790c7b23d43f88d9ef2f97dab3f7ac9"
server_key_iv: "4a39a4c89c1d8a679e238165d85824b1"
server_key_version: "encryptionkey@0"
I20260810 11:47:22.877709 2940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:22.891769 2940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:22.896137 2940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:22.896468 2940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:22.916320 2940 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38645
I20260810 11:47:22.916360 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38645 every 8 connection(s)
I20260810 11:47:22.917909 2940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:22.921977 3016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:22.924577 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 2940
I20260810 11:47:22.924846 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:47:22.925696 20854 external_mini_cluster.cc:1428] Setting key bdbaed981769d2a7b4d8d357811550e3
I20260810 11:47:22.932209 3016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:47:22 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.93.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.189@KRBTEST.COM" created.
I20260810 11:47:22.961308 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:22.936752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.20.93.190:38645)
0810 11:47:22.938917 (+ 2165us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:22.938927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:22.938972 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:47:22.941185 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:22.941197 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:22.941552 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:22.941814 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:22.943134 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:22.945228 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:22.946858 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:22.947952 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:22.948195 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:22.952046 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:22.952058 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:22.952081 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:22.952131 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:22.956518 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:22.957460 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:22.957468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:22.957470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:22.957703 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:22.958214 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:22.958218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:22.958220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:22.958826 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:22.958995 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:22.959691 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 11:47:22.960239 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":1961,"spinlock_wait_cycles":76672,"thread_start_us":1704,"threads_started":1}
I20260810 11:47:22.961412 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:22.936752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38645 (local address 127.0.0.1:42968)
0810 11:47:22.937844 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:22.937899 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:22.940711 (+ 2812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:22.941884 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:22.941891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:22.942528 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:22.942900 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:22.942919 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:22.945433 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:22.945438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:22.946685 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:22.946700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:22.948243 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:22.948249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:22.948382 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:22.949180 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:22.949205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:22.951741 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:22.956716 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:22.956728 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:22.956740 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:22.957284 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:22.957819 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:22.957823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:22.957825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:22.958028 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:22.958981 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:22.958988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:22.959280 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 11:47:22.959512 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 11:47:22.960002 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":476,"threads_started":1}
I20260810 11:47:22.964463 3016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:47:22.965705 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:22.965123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33892)
0810 11:47:22.965349 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:22.965505 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 11:47:22.966039 2958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
I20260810 11:47:22.967841 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:22.967477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33898)
0810 11:47:22.967628 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:22.967700 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:47:22.968159 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:39945: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
I20260810 11:47:22.997331 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:39945
--webserver_interface=127.20.93.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:23.017710 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } attempt: 1
I20260810 11:47:23.019048 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.018544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33912)
0810 11:47:23.018780 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.018899 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 11:47:23.020164 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.019755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33928)
0810 11:47:23.019908 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.019989 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:47:23.020356 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:39945: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
I20260810 11:47:23.088600 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } attempt: 2
I20260810 11:47:23.089919 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.089322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33932)
0810 11:47:23.089531 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.089712 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:47:23.091046 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.090646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33942)
0810 11:47:23.090806 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.090904 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:47:23.091238 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:39945: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
I20260810 11:47:23.182487 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } attempt: 3
I20260810 11:47:23.183622 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.183186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33956)
0810 11:47:23.183385 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.183487 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 11:47:23.184634 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.184260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33970)
0810 11:47:23.184403 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.184501 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 11:47:23.184836 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:39945: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
I20260810 11:47:23.328118 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } attempt: 4
I20260810 11:47:23.329417 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.328843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33978)
0810 11:47:23.329176 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.329268 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260810 11:47:23.330462 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.330094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:33990)
0810 11:47:23.330244 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.330341 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:47:23.330641 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:39945: Network error: Client connection negotiation failed: client connection to 127.20.93.189:39945: connect: Connection refused (error 111)
W20260810 11:47:23.357718 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:23.358120 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:23.358291 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:23.366006 3024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:23.366178 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:23.366293 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:23.366402 3024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:23.366497 3024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:23.377300 3024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.93.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:23.378891 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:23.380276 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:23.392764 3030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:23.392835 3033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:23.392870 3031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:23.393703 3024 server_base.cc:1034] running on GCE node
I20260810 11:47:23.394497 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:23.396008 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:23.397280 3024 hybrid_clock.cc:648] HybridClock initialized: now 1786362443397237 us; error 63 us; skew 500 ppm
I20260810 11:47:23.397739 3024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:23 dist-test-slave-njxd krb5kdc[2926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:23.401072 3024 init.cc:377] Logged in from keytab as kudu/127.20.93.189@KRBTEST.COM (short username kudu)
I20260810 11:47:23.402565 3024 webserver.cc:460] Webserver started at http://127.20.93.189:45929/ using document root <none> and password file <none>
I20260810 11:47:23.403244 3024 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:23.403348 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:23.403604 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:23.406704 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data/instance:
uuid: "27b44945c34849bbbb3679f32687c377"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "17209be2fea8f11ef6fb70c7f7cde45a"
server_key_iv: "75036c7098fe8e222433bdde08e50246"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.407939 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal/instance:
uuid: "27b44945c34849bbbb3679f32687c377"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "17209be2fea8f11ef6fb70c7f7cde45a"
server_key_iv: "75036c7098fe8e222433bdde08e50246"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.413412 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:47:23.417021 3041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:23.418475 3024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:23.418679 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "27b44945c34849bbbb3679f32687c377"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "17209be2fea8f11ef6fb70c7f7cde45a"
server_key_iv: "75036c7098fe8e222433bdde08e50246"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.418915 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:23.442452 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:23.447484 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:23.447826 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:23.467680 3024 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.189:39945
I20260810 11:47:23.467717 3095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.189:39945 every 8 connection(s)
I20260810 11:47:23.469167 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
I20260810 11:47:23.473345 3096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:23.477831 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3024
I20260810 11:47:23.478063 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-1/wal/instance
I20260810 11:47:23.478629 20854 external_mini_cluster.cc:1428] Setting key 3d0ab1c8d482db34dcd15aeddde7ce70
I20260810 11:47:23.481778 3096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:47:23 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:23.508318 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.485352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.20.93.190:38645)
0810 11:47:23.485905 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:23.485912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:23.487116 (+ 1204us) server_negotiation.cc:407] Connection header received
0810 11:47:23.490120 (+ 3004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:23.490125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:23.490199 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:23.490308 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:23.491991 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.493240 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.495034 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.495818 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.495968 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.500139 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:23.500149 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:23.500156 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:23.500220 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:23.504309 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.505176 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.505185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.505192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.505383 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.505921 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.505929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.505935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.506373 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:23.506547 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:23.507547 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0810 11:47:23.507817 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":284,"threads_started":1}
WARNING: no policy specified for kudu/127.20.93.188@KRBTEST.COM; defaulting to no policy
I20260810 11:47:23.509003 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.486019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38645 (local address 127.0.0.1:42972)
0810 11:47:23.486920 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.486975 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:23.489785 (+ 2810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:23.490624 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:23.490634 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:23.491192 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:23.491790 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.491807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.493435 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.493448 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.494872 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.494884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.496052 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.496057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.496272 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.497071 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:23.497095 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:23.499838 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:23.504478 (+ 4640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.504486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.504501 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.504992 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.505515 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.505519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.505522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.505782 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.506530 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:23.506538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:23.507087 (+ 549us) client_negotiation.cc:769] Sending connection context
0810 11:47:23.507379 (+ 292us) client_negotiation.cc:240] Negotiation successful
0810 11:47:23.507790 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":340,"threads_started":1}
I20260810 11:47:23.511060 3096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.188@KRBTEST.COM" created.
Aug 10 11:47:23 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:47:23.528586 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.511867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:34002)
0810 11:47:23.512045 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.512067 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:23.512183 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:23.513428 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:23.513435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:23.514024 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:23.514265 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.514281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.515676 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.515685 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.517184 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.517214 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.518150 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.518159 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.518432 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.519074 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:23.519103 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:23.521959 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:23.525948 (+ 3989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.525953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.525958 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.526344 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.526745 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.526750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.526753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.526915 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.527545 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:23.527550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:23.527707 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 11:47:23.527852 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 11:47:23.528150 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 11:47:23.528632 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.512178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.20.93.189:39945)
0810 11:47:23.512967 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:23.512973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:23.512999 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:47:23.513043 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:23.513049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:23.513238 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:23.513359 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:23.514441 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.515516 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.517348 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.517982 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.518082 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.522178 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:23.522191 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:23.522207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:23.522243 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:23.525804 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.526488 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.526498 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.526501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.526630 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.527035 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.527040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.527043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.527419 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:23.527653 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:23.528033 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 11:47:23.528243 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":483,"threads_started":1}
I20260810 11:47:23.531502 3096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:47:23.532727 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.532231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35792)
0810 11:47:23.532443 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.532560 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 11:47:23.533037 3042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.534821 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.534437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35804)
0810 11:47:23.534612 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.534695 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:47:23.535033 3096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
I20260810 11:47:23.550011 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:36011
--webserver_interface=127.20.93.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:23.584347 3096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 1
I20260810 11:47:23.585670 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.585127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35818)
0810 11:47:23.585334 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.585449 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 11:47:23.586797 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.586382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35824)
0810 11:47:23.586569 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.586651 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:47:23.587085 3096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.629933 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } attempt: 5
Aug 10 11:47:23 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:47:23.645684 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.631003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.20.93.189:39945)
0810 11:47:23.631173 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:23.631178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:23.631198 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:47:23.631292 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:23.631296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:23.631377 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:23.631462 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:23.632605 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.633309 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.635249 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.635693 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.635792 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.639931 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:23.639937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:23.639940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:23.639977 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:23.642911 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.643607 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.643613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.643616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.643750 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:23.644146 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:23.644152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:23.644155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:23.644463 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:23.644628 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:23.645058 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 11:47:23.645301 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:47:23.645679 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.630752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:34008)
0810 11:47:23.630979 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.631002 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:23.631160 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:23.631542 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:23.631547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:23.632166 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:23.632427 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.632441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.633462 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.633470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.635091 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:23.635113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:23.635877 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:23.635883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:23.636120 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:23.636826 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:23.636851 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:23.639692 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:23.643066 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.643072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.643077 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.643474 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.643873 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:23.643878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:23.643882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:23.644028 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:23.644589 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:23.644594 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:23.644765 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 11:47:23.644952 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 11:47:23.645266 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:47:23.646975 3016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:47:23.648120 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.647696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35838)
0810 11:47:23.647886 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.647969 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:47:23.649119 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.648778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35844)
0810 11:47:23.648932 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.649015 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:47:23.649360 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.655313 3096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 2
I20260810 11:47:23.656432 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.655972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35858)
0810 11:47:23.656168 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.656273 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:47:23.657480 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.657113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35862)
0810 11:47:23.657285 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.657355 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:47:23.657733 3096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.700608 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 1
I20260810 11:47:23.701923 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.701363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35876)
0810 11:47:23.701625 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.701731 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 11:47:23.703011 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.702664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35888)
0810 11:47:23.702836 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.702904 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:47:23.703210 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.748987 3096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 3
I20260810 11:47:23.750414 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.749854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35904)
0810 11:47:23.750131 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.750233 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 11:47:23.751600 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.751157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35906)
0810 11:47:23.751392 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.751469 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260810 11:47:23.751811 3096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.771509 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 2
I20260810 11:47:23.773020 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.772415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35916)
0810 11:47:23.772711 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.772821 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 11:47:23.774242 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.773856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35922)
0810 11:47:23.774019 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.774127 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:47:23.774464 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.880733 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 3
I20260810 11:47:23.882189 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.881571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35938)
0810 11:47:23.881819 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.881925 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 11:47:23.883369 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.882966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35942)
0810 11:47:23.883165 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.883248 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 11:47:23.883587 3016 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
I20260810 11:47:23.895022 3096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 4
I20260810 11:47:23.896188 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.895678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35946)
0810 11:47:23.895930 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.896021 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 11:47:23.897156 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:23.896828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35956)
0810 11:47:23.896979 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:23.897054 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:47:23.897382 3096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:36011: Network error: Client connection negotiation failed: client connection to 127.20.93.188:36011: connect: Connection refused (error 111)
W20260810 11:47:23.927421 3104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:23.927866 3104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:23.928017 3104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:23.935814 3104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:23.935945 3104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:23.936023 3104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:23.936112 3104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:23.936200 3104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:23.946769 3104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38645,127.20.93.189:39945,127.20.93.188:36011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:36011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.93.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:23.948210 3104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:23.949669 3104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:23.962634 3112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:23.962738 3109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:23.962764 3110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:23.963649 3104 server_base.cc:1034] running on GCE node
I20260810 11:47:23.964486 3104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:23.966131 3104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:23.967345 3104 hybrid_clock.cc:648] HybridClock initialized: now 1786362443967294 us; error 38 us; skew 500 ppm
I20260810 11:47:23.967768 3104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:23 dist-test-slave-njxd krb5kdc[2926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:23.971154 3104 init.cc:377] Logged in from keytab as kudu/127.20.93.188@KRBTEST.COM (short username kudu)
I20260810 11:47:23.972839 3104 webserver.cc:460] Webserver started at http://127.20.93.188:46193/ using document root <none> and password file <none>
I20260810 11:47:23.973660 3104 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:23.973812 3104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:23.974143 3104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:23.977329 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data/instance:
uuid: "1a6c488135bd43baa0b0f8f38e4511c2"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "53d20eadaf8bbbfa02db387458852427"
server_key_iv: "bbb4d96c05fdb0c4c1050ff763f82979"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.978252 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal/instance:
uuid: "1a6c488135bd43baa0b0f8f38e4511c2"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "53d20eadaf8bbbfa02db387458852427"
server_key_iv: "bbb4d96c05fdb0c4c1050ff763f82979"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.983692 3104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:47:23.987310 3120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:23.988782 3104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:47:23.988977 3104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "1a6c488135bd43baa0b0f8f38e4511c2"
format_stamp: "Formatted at 2026-08-10 11:47:23 on dist-test-slave-njxd"
server_key: "53d20eadaf8bbbfa02db387458852427"
server_key_iv: "bbb4d96c05fdb0c4c1050ff763f82979"
server_key_version: "encryptionkey@0"
I20260810 11:47:23.989187 3104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:24.008345 3104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:24.013243 3104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:24.013594 3104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:24.032967 3104 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.188:36011
I20260810 11:47:24.032999 3178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.188:36011 every 8 connection(s)
I20260810 11:47:24.034425 3104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
I20260810 11:47:24.038285 3179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:24.040657 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3104
I20260810 11:47:24.040869 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786362228785499-20854-0/minicluster-data/master-2/wal/instance
I20260810 11:47:24.041498 20854 external_mini_cluster.cc:1428] Setting key 79f8248785a191d028f1125e72af0e0d
I20260810 11:47:24.046640 3179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:24.060818 3016 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 4
I20260810 11:47:24.067138 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.049857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.20.93.190:38645)
0810 11:47:24.051009 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.051015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.051039 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:47:24.051090 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.051093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.051161 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.051249 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.052710 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.053843 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.055834 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.056215 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.056325 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.060001 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.060007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.060010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.060048 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.063718 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.064573 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.064579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.064583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.064712 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.065215 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.065222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.065236 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.065672 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.065820 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.066466 (+ 646us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.066679 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":877,"threads_started":1}
I20260810 11:47:24.067664 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.049726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38645 (local address 127.0.0.1:42976)
0810 11:47:24.050591 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:24.050647 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:24.050837 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:24.051356 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:24.051364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:24.052186 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:24.052551 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.052564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.054012 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.054017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.055673 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.055690 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.056471 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.056480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.056687 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.057296 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:24.057319 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:24.059800 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:24.063842 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.063855 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.063869 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.064455 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.064832 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.064837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.064840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.065047 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.065909 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:24.065919 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:24.066107 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 11:47:24.066378 (+ 271us) client_negotiation.cc:240] Negotiation successful
0810 11:47:24.066680 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"spinlock_wait_cycles":14720,"thread_start_us":309,"threads_started":1}
I20260810 11:47:24.067771 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.043495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.20.93.188:36011)
0810 11:47:24.044338 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.044350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.044392 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:47:24.046203 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.046228 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.046612 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.046961 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.048830 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.050849 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.052836 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.053842 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.054030 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.058940 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.058948 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.058960 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.058988 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.062796 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.063570 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.063579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.063588 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.063794 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.064247 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.064254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.064258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.064978 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.065291 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.065808 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.066233 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":412,"threads_started":1}
I20260810 11:47:24.070220 3179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:47:24.080410 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.061696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.20.93.188:36011)
0810 11:47:24.062502 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.062512 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.062537 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:47:24.062585 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.062590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.062693 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.062823 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.063828 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.065109 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.067051 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.067783 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.068369 (+ 586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.072164 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.072174 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.072180 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.072216 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.077107 (+ 4891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.078008 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.078014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.078018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.078183 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.078641 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.078648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.078656 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.079056 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.079211 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.079720 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.079947 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":452,"threads_started":1}
I20260810 11:47:24.080669 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.061476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35976)
0810 11:47:24.061794 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:24.061819 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:24.061923 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:24.062876 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:24.062882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:24.063415 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:24.063650 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.063663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.065272 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.065279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.066850 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.066865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.067966 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.067972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.068135 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.068681 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:24.068697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:24.071901 (+ 3204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:24.077288 (+ 5387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.077295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.077299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.077856 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.078307 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.078314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.078318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.078494 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.079206 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:24.079212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:24.079409 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 11:47:24.079571 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 11:47:24.079884 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:24.085834 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.070960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:39945 (local address 127.0.0.1:34016)
0810 11:47:24.071145 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:24.071160 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:24.071286 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:24.071733 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:24.071740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:24.072221 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:24.072432 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.072444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.073742 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.073747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.074833 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.074846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.075641 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.075646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.075884 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.076451 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:24.076468 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:24.079151 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:24.083366 (+ 4215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.083372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.083376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.083761 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.084164 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.084169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.084173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.084334 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.084842 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:24.084847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:24.084998 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 11:47:24.085155 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 11:47:24.085452 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 11:47:24.085914 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.071161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.20.93.189:39945)
0810 11:47:24.071383 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.071389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.071405 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:47:24.071465 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.071469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.071563 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.071668 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.072593 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.073555 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.074990 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.075485 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.075618 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.079421 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.079427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.079431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.079466 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.083228 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.083892 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.083898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.083902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.084047 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.084447 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.084453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.084456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.084711 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.084885 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.085249 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.085461 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:47:24.085958 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: Bootstrap starting.
I20260810 11:47:24.087304 3179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:47:24.090340 3016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:24.091568 3016 log.cc:824] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:24.093601 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.072288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.20.93.190:38645)
0810 11:47:24.072456 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.072460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.072474 (+ 14us) server_negotiation.cc:407] Connection header received
0810 11:47:24.072532 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.072538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.072615 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.072719 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.074497 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.075164 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.077176 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.077913 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.078054 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.086793 (+ 8739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.086798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.086801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.086830 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.090570 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.091419 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.091424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.091428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.091592 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.092067 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.092073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.092076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.092444 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.092760 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.093061 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.093283 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:47:24.098956 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: No bootstrap required, opened a new log
I20260810 11:47:24.105029 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.088056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35982)
0810 11:47:24.088329 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:24.088349 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:24.088445 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:24.088769 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:24.088773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:24.089132 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:24.089260 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.089269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.090321 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.090329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.091602 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.091614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.093006 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.093014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.093318 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.094120 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:24.094159 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:24.097405 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:24.101439 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.101447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.101452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.102380 (+ 928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.102841 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.102846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.102850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.103032 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.103737 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:24.103743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:24.103952 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 11:47:24.104140 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 11:47:24.104538 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260810 11:47:24.105136 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.088229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.20.93.188:36011)
0810 11:47:24.088400 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.088408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.088426 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:47:24.088545 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.088551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.088646 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.088745 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.089409 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.090182 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.091764 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.092657 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.092798 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.097669 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.097676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.097684 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.097723 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.101285 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.102522 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.102529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.102534 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.102698 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.103170 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.103175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.103178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.103590 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.103749 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.104273 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.104767 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":179,"server-negotiator.queue_time_us":67}
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:47:24.109851 3179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: Bootstrap starting.
I20260810 11:47:24.110955 3016 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.111454 3016 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:24.111589 3016 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c5c52b301e46198b7a75b2cd501aed, State: Initialized, Role: FOLLOWER
I20260810 11:47:24.112531 3016 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.113834 3179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:24.113685 3188 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.114323 3188 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.114732 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.095437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.20.93.189:39945)
0810 11:47:24.095606 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.095612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.095685 (+ 73us) server_negotiation.cc:407] Connection header received
0810 11:47:24.095881 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.095887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.095967 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.096071 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.097929 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.098925 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.100444 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.100902 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.101052 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.106862 (+ 5810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.106871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.106875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.106922 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.110710 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.111896 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.111905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.111911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.112121 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.112660 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.112667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.112672 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.113075 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.113374 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.114051 (+ 677us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.114328 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:47:24.115172 3016 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:24.115342 3179 log.cc:824] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:24.119402 3179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: No bootstrap required, opened a new log
I20260810 11:47:24.121160 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:47:24.136081 3179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.136847 3179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:24.137082 3179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6c488135bd43baa0b0f8f38e4511c2, State: Initialized, Role: FOLLOWER
I20260810 11:47:24.138188 3179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.141363 3207 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:47:24.144372 3209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.145308 3209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.146363 3179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:47:24.149449 2963 tablet.cc:2369] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:24.170588 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.127035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42998 (local address 127.20.93.190:38645)
0810 11:47:24.127257 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.127266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.134537 (+ 7271us) server_negotiation.cc:407] Connection header received
0810 11:47:24.135900 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.135906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.136004 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.136092 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.142229 (+ 6137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.143392 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.153189 (+ 9797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.154003 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.154167 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.160666 (+ 6499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.160673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.160679 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.160723 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.166328 (+ 5605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.167334 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.167342 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.167347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.167505 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.168019 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.168024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.168027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.168414 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.169710 (+ 1296us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.169885 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.170163 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:47:24.173951 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.130186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.20.93.189:39945)
0810 11:47:24.130453 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.130461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.140042 (+ 9581us) server_negotiation.cc:407] Connection header received
0810 11:47:24.140549 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.140557 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.140653 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.140735 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.146025 (+ 5290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.147125 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.155339 (+ 8214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.156097 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.156246 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.163108 (+ 6862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.163116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.163123 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.163176 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.170209 (+ 7033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.171280 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.171288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.171293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.171460 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.172004 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.172009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.172012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.172409 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.172677 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.173307 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.173567 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:47:24.175565 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.134779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.20.93.188:36011)
0810 11:47:24.134991 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.135000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.140332 (+ 5332us) server_negotiation.cc:407] Connection header received
0810 11:47:24.140588 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.140593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.140684 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.140758 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.149650 (+ 8892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.150770 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.156429 (+ 5659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.157282 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.157432 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.164534 (+ 7102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.164543 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.164548 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.164590 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.168501 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.172099 (+ 3598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.172105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.172108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.172277 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.172794 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.172799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.172802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.173187 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.174578 (+ 1391us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.174759 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.175027 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:47:24.166886 3104 master_runner.cc:186] Error getting master registration for 127.20.93.189:39945: OK, instance_id { permanent_uuid: "27b44945c34849bbbb3679f32687c377" instance_seqno: 1786362443461737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:47:24.183848 3104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:47:24.151170 2940 master_runner.cc:186] Error getting master registration for 127.20.93.189:39945: OK, instance_id { permanent_uuid: "27b44945c34849bbbb3679f32687c377" instance_seqno: 1786362443461737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:47:24.185019 2940 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 11:47:24.186679 3221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:47:24.186836 3221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:47:24.196640 3096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } attempt: 5
Aug 10 11:47:24 dist-test-slave-njxd krb5kdc[2926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362443, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:47:24.214781 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.197275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:36011 (local address 127.0.0.1:35994)
0810 11:47:24.197499 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:24.197521 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:24.197666 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:24.197981 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:24.197986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:24.198575 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:24.198820 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.198837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.199975 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.199984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.201462 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:24.201480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.202330 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.202337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:24.202578 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.203349 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:24.203375 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:24.206379 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:24.210595 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.210599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.210602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.211102 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.211640 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:24.211644 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:24.211647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:24.211810 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:24.212739 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:24.212747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:24.212948 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 11:47:24.213990 (+ 1042us) client_negotiation.cc:240] Negotiation successful
0810 11:47:24.214357 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 11:47:24.214860 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:24.197500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.20.93.188:36011)
0810 11:47:24.197695 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:24.197700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:24.197722 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:47:24.197776 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:24.197780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:24.197836 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:24.197914 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:24.199027 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.199823 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.201673 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:24.202176 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:24.202281 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:24.206597 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:24.206604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:24.206609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:24.206651 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:24.210424 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.211240 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.211246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.211250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.211432 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:24.211934 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:24.211939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:24.211943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:24.212352 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:24.212519 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:24.214218 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0810 11:47:24.214476 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 11:47:24.219838 3096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: Bootstrap starting.
I20260810 11:47:24.223569 3096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:24.225133 3096 log.cc:824] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:24.227993 3096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: No bootstrap required, opened a new log
I20260810 11:47:24.236554 3096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.236902 3096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:24.237042 3096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b44945c34849bbbb3679f32687c377, State: Initialized, Role: FOLLOWER
I20260810 11:47:24.237744 3096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.238731 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.239246 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.240113 3096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:47:24.251750 3235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:47:24.252022 3235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:47:24.256019 3024 master_runner.cc:263] No leader master found from master 27b44945c34849bbbb3679f32687c377
I20260810 11:47:24.256192 3024 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 11:47:24.265851 3127 tablet.cc:2369] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:24.378693 3209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:24.379040 3209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.380813 3209 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4c5c52b301e46198b7a75b2cd501aed (127.20.93.190:38645), 27b44945c34849bbbb3679f32687c377 (127.20.93.189:39945)
I20260810 11:47:24.381563 2988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c5c52b301e46198b7a75b2cd501aed" is_pre_election: true
I20260810 11:47:24.381866 3069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b44945c34849bbbb3679f32687c377" is_pre_election: true
I20260810 11:47:24.382297 2988 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6c488135bd43baa0b0f8f38e4511c2 in term 0.
I20260810 11:47:24.382485 3069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6c488135bd43baa0b0f8f38e4511c2 in term 0.
I20260810 11:47:24.382969 3125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6c488135bd43baa0b0f8f38e4511c2, a4c5c52b301e46198b7a75b2cd501aed; no voters:
I20260810 11:47:24.383422 3209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:47:24.383580 3209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:47:24.383677 3209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:24.387493 3209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.388250 3209 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [CANDIDATE]: Term 1 election: Requested vote from peers a4c5c52b301e46198b7a75b2cd501aed (127.20.93.190:38645), 27b44945c34849bbbb3679f32687c377 (127.20.93.189:39945)
I20260810 11:47:24.388795 3069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b44945c34849bbbb3679f32687c377"
I20260810 11:47:24.389041 3069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:24.389503 2988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c5c52b301e46198b7a75b2cd501aed"
I20260810 11:47:24.389784 2988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:24.392735 3069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a6c488135bd43baa0b0f8f38e4511c2 in term 1.
I20260810 11:47:24.393256 3121 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6c488135bd43baa0b0f8f38e4511c2, 27b44945c34849bbbb3679f32687c377; no voters:
I20260810 11:47:24.393294 2988 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a6c488135bd43baa0b0f8f38e4511c2 in term 1.
I20260810 11:47:24.393697 3209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:24.394376 3209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1a6c488135bd43baa0b0f8f38e4511c2, State: Running, Role: LEADER
I20260810 11:47:24.394997 3209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } }
I20260810 11:47:24.400628 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6c488135bd43baa0b0f8f38e4511c2. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.401086 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:24.401988 3241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:24.404273 3241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:24.413698 3069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [term 1 FOLLOWER]: Refusing update from remote peer 1a6c488135bd43baa0b0f8f38e4511c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:24.413938 2988 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [term 1 FOLLOWER]: Refusing update from remote peer 1a6c488135bd43baa0b0f8f38e4511c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:24.414913 3238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:24.415617 3238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:24.444265 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6c488135bd43baa0b0f8f38e4511c2. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.444618 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.449755 3244 mvcc.cc:205] Tried to move back new op lower bound from 7316940572307476480 to 7316940572252286976. Current Snapshot: MvccSnapshot[applied={T|T < 7316940572307476480}]
I20260810 11:47:24.457432 3188 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6c488135bd43baa0b0f8f38e4511c2. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.457842 3188 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.461665 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.461972 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.463860 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.464135 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:24.464407 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.464627 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:24.470366 3241 catalog_manager.cc:1347] Generated new cluster ID: 6e2b886f0d7e40b69192a9afcf26f9a6
I20260810 11:47:24.470503 3241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:24.481860 3188 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4c5c52b301e46198b7a75b2cd501aed" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38645 } } peers { permanent_uuid: "27b44945c34849bbbb3679f32687c377" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 39945 } } peers { permanent_uuid: "1a6c488135bd43baa0b0f8f38e4511c2" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 36011 } } }
I20260810 11:47:24.482192 3188 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:47:24.493252 3241 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:47:24.495363 3241 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:24.509631 3241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a6c488135bd43baa0b0f8f38e4511c2: Generated new TSK 0
I20260810 11:47:24.510823 3241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:47:25.146111 3207 catalog_manager.cc:1259] Loaded cluster ID: 6e2b886f0d7e40b69192a9afcf26f9a6
I20260810 11:47:25.146256 3207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: loading cluster ID for follower catalog manager: success
I20260810 11:47:25.149338 3207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: acquiring CA information for follower catalog manager: success
I20260810 11:47:25.151458 3207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4c5c52b301e46198b7a75b2cd501aed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:47:25.254391 3235 catalog_manager.cc:1259] Loaded cluster ID: 6e2b886f0d7e40b69192a9afcf26f9a6
I20260810 11:47:25.254556 3235 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: loading cluster ID for follower catalog manager: success
I20260810 11:47:25.257222 3235 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: acquiring CA information for follower catalog manager: success
I20260810 11:47:25.259406 3235 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 27b44945c34849bbbb3679f32687c377: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:47:25.620000 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 2940
I20260810 11:47:25.621977 3011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:25.738438 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 2940
I20260810 11:47:25.765377 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3024
I20260810 11:47:25.766681 3090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:25.878185 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3024
I20260810 11:47:25.903131 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3104
I20260810 11:47:25.904402 3174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:26.015967 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3104
2026-08-10T11:47:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5087 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:47:26 dist-test-slave-njxd krb5kdc[3268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:47:26 dist-test-slave-njxd krb5kdc[3268](info): set up 2 sockets
Aug 10 11:47:26 dist-test-slave-njxd krb5kdc[3268](info): commencing operation
krb5kdc: starting...
W20260810 11:47:27.207111 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:47:27 dist-test-slave-njxd krb5kdc[3268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:47:27.521924 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44439
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:27.883913 3281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:27.884246 3281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:27.884342 3281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:27.892109 3281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:27.892251 3281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:27.892311 3281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:27.892366 3281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:27.892410 3281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:27.903193 3281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:27.904575 3281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:27.906011 3281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:27.918660 3287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:27.918565 3290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:27.918622 3288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:27.919644 3281 server_base.cc:1034] running on GCE node
I20260810 11:47:27.920588 3281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:27.922271 3281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:27.923547 3281 hybrid_clock.cc:648] HybridClock initialized: now 1786362447923501 us; error 54 us; skew 500 ppm
I20260810 11:47:27.923961 3281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:27 dist-test-slave-njxd krb5kdc[3268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362447, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:27.933435 3281 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:27.935256 3281 webserver.cc:460] Webserver started at http://127.20.93.190:40979/ using document root <none> and password file <none>
I20260810 11:47:27.936055 3281 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:27.936221 3281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:27.936568 3281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:27.940150 3281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "46f486a1a2454f67b8c674feba76f699"
format_stamp: "Formatted at 2026-08-10 11:47:27 on dist-test-slave-njxd"
server_key: "f5923092831fbc12dad51181e5719013"
server_key_iv: "7962efb94651d497c2b4074d730b4799"
server_key_version: "encryptionkey@0"
I20260810 11:47:27.941100 3281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "46f486a1a2454f67b8c674feba76f699"
format_stamp: "Formatted at 2026-08-10 11:47:27 on dist-test-slave-njxd"
server_key: "f5923092831fbc12dad51181e5719013"
server_key_iv: "7962efb94651d497c2b4074d730b4799"
server_key_version: "encryptionkey@0"
I20260810 11:47:27.946959 3281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 11:47:27.950887 3299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:27.953328 3281 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:47:27.953485 3281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "46f486a1a2454f67b8c674feba76f699"
format_stamp: "Formatted at 2026-08-10 11:47:27 on dist-test-slave-njxd"
server_key: "f5923092831fbc12dad51181e5719013"
server_key_iv: "7962efb94651d497c2b4074d730b4799"
server_key_version: "encryptionkey@0"
I20260810 11:47:27.953735 3281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:27.969849 3281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:27.975096 3281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:27.975389 3281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:27.995911 3281 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44439
I20260810 11:47:27.995935 3354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44439 every 8 connection(s)
I20260810 11:47:27.997418 3281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:28.001410 3355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:28.003600 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3281
I20260810 11:47:28.003861 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:47:28.004527 20854 external_mini_cluster.cc:1428] Setting key dfb81ab8a9359638f0ff3babcf5bba39
I20260810 11:47:28.013995 3355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699: Bootstrap starting.
I20260810 11:47:28.018074 3355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:28.019493 3355 log.cc:824] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:47:28 dist-test-slave-njxd krb5kdc[3268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:28.023005 3355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699: No bootstrap required, opened a new log
I20260810 11:47:28.031714 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:28.006563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.20.93.190:44439)
0810 11:47:28.007433 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:28.007445 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:28.007490 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:47:28.009319 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:28.009343 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:28.009844 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:28.010235 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:28.012340 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:28.013945 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:28.015886 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:28.016668 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:28.016815 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:28.022487 (+ 5672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:28.022501 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:28.022521 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:28.022565 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:28.026807 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:28.027565 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:28.027573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:28.027582 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:28.027826 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:28.028218 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:28.028223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:28.028226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:28.028981 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:28.029305 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:28.029692 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 11:47:28.030187 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":413,"threads_started":1}
I20260810 11:47:28.033845 3355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } }
I20260810 11:47:28.034301 3355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:28.034430 3355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f486a1a2454f67b8c674feba76f699, State: Initialized, Role: FOLLOWER
I20260810 11:47:28.035117 3355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } }
I20260810 11:47:28.035395 3355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:28.035547 3355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:28.035725 3355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:28.036123 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:47:28.039090 3355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } }
I20260810 11:47:28.039693 3355 leader_election.cc:302] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46f486a1a2454f67b8c674feba76f699; no voters:
I20260810 11:47:28.040421 3355 leader_election.cc:288] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:47:28.040544 3361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:28.041358 3361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [term 1 LEADER]: Becoming Leader. State: Replica: 46f486a1a2454f67b8c674feba76f699, State: Running, Role: LEADER
I20260810 11:47:28.041941 3361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } }
I20260810 11:47:28.042706 3355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:28.044873 3363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f486a1a2454f67b8c674feba76f699. Latest consensus state: current_term: 1 leader_uuid: "46f486a1a2454f67b8c674feba76f699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } } }
I20260810 11:47:28.045374 3363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:28.047981 3362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46f486a1a2454f67b8c674feba76f699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f486a1a2454f67b8c674feba76f699" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44439 } } }
I20260810 11:47:28.048348 3362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:28.048642 3370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:28.054279 3370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:28.063516 3370 catalog_manager.cc:1347] Generated new cluster ID: 787fe951fad04cbab620909754bd12e0
I20260810 11:47:28.063644 3370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:28.080224 3370 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:47:28.082381 3370 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:28.099869 3370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46f486a1a2454f67b8c674feba76f699: Generated new TSK 0
I20260810 11:47:28.101174 3370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:47:28.150277 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3281
I20260810 11:47:28.151638 3350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:28.251870 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3281
2026-08-10T11:47:28Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2231 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:47:28 dist-test-slave-njxd krb5kdc[3385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:47:28 dist-test-slave-njxd krb5kdc[3385](info): set up 2 sockets
Aug 10 11:47:28 dist-test-slave-njxd krb5kdc[3385](info): commencing operation
krb5kdc: starting...
W20260810 11:47:29.454319 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:47:29 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:47:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:47:29.767745 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:42899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:30.129860 3400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:30.130200 3400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:30.130378 3400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:30.138046 3400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:30.138173 3400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:30.138278 3400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:30.138365 3400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:30.138450 3400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:30.148870 3400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:42899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:30.150370 3400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:30.151778 3400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:30.164880 3407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:30.164852 3408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:30.165021 3410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:30.165624 3400 server_base.cc:1034] running on GCE node
I20260810 11:47:30.166952 3400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:30.168586 3400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:30.169806 3400 hybrid_clock.cc:648] HybridClock initialized: now 1786362450169717 us; error 74 us; skew 500 ppm
I20260810 11:47:30.170239 3400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:30 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362450, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:30.173810 3400 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:30.175444 3400 webserver.cc:460] Webserver started at http://127.20.93.190:43073/ using document root <none> and password file <none>
I20260810 11:47:30.176177 3400 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:30.176293 3400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:30.176565 3400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:30.179873 3400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "f4dd5b63ad984fbcba214cb5b2647db8"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "fb2775441450868f5c0c729102951c07"
server_key_iv: "afaa6497c1145e3306e136164e8ce88e"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.180673 3400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "f4dd5b63ad984fbcba214cb5b2647db8"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "fb2775441450868f5c0c729102951c07"
server_key_iv: "afaa6497c1145e3306e136164e8ce88e"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.186426 3400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:47:30.190225 3418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:30.191735 3400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 11:47:30.191946 3400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "f4dd5b63ad984fbcba214cb5b2647db8"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "fb2775441450868f5c0c729102951c07"
server_key_iv: "afaa6497c1145e3306e136164e8ce88e"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.192210 3400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:30.206998 3400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:30.211710 3400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:30.211987 3400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:30.231446 3400 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:42899
I20260810 11:47:30.231495 3475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:42899 every 8 connection(s)
I20260810 11:47:30.233073 3400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:30.237478 3476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:30.238711 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3400
I20260810 11:47:30.238988 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:47:30.239750 20854 external_mini_cluster.cc:1428] Setting key d10d5f6e3e7aaca5762658bb28bf362d
I20260810 11:47:30.253954 3476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap starting.
Aug 10 11:47:30 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:30.258143 3476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:30.259593 3476 log.cc:824] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:30.263695 3476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: No bootstrap required, opened a new log
I20260810 11:47:30.267630 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:30.242128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.20.93.190:42899)
0810 11:47:30.243072 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:30.243091 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:30.243163 (+ 72us) server_negotiation.cc:407] Connection header received
0810 11:47:30.245030 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:30.245066 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:30.245715 (+ 649us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:30.246244 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:30.247770 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.249975 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.251583 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.252761 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.252977 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:30.258214 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:30.258257 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:30.258283 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:30.258359 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:30.262311 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:30.263435 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:30.263443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:30.263449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:30.263649 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:30.264220 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:30.264224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:30.264226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:30.264877 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:30.265161 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:30.265858 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 11:47:30.266385 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":386,"threads_started":1}
I20260810 11:47:30.273785 3476 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:30.274279 3476 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:30.274405 3476 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Initialized, Role: FOLLOWER
I20260810 11:47:30.275182 3476 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:30.275465 3476 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:30.275636 3476 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:30.275842 3476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:30.278344 3476 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:30.278946 3476 leader_election.cc:302] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4dd5b63ad984fbcba214cb5b2647db8; no voters:
I20260810 11:47:30.279762 3476 leader_election.cc:288] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:47:30.279922 3482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:30.280910 3482 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 LEADER]: Becoming Leader. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Running, Role: LEADER
I20260810 11:47:30.281631 3482 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:30.282246 3476 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:30.286276 3484 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4dd5b63ad984fbcba214cb5b2647db8. Latest consensus state: current_term: 1 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:30.286237 3483 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:30.286761 3484 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:30.286810 3483 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:30.293610 3493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:30.301398 3493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:30.311322 3493 catalog_manager.cc:1347] Generated new cluster ID: 208d0898738b4d4184439bf03807b42c
I20260810 11:47:30.311538 3493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:30.352558 3493 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:47:30.354290 3493 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:30.363523 3493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Generated new TSK 0
I20260810 11:47:30.364476 3493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:47:30.438271 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:30.811270 3506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:30.811645 3506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:30.811826 3506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:30.819545 3506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:30.819732 3506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:30.819916 3506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:30.830104 3506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:30.831559 3506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:30.832979 3506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:30.847322 3512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:30.847301 3513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:30.847246 3515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:30.848855 3506 server_base.cc:1034] running on GCE node
I20260810 11:47:30.849704 3506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:30.850900 3506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:30.852161 3506 hybrid_clock.cc:648] HybridClock initialized: now 1786362450852086 us; error 70 us; skew 500 ppm
I20260810 11:47:30.852545 3506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:30 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362450, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:30.856211 3506 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:30.858109 3506 webserver.cc:460] Webserver started at http://127.20.93.129:40743/ using document root <none> and password file <none>
I20260810 11:47:30.858939 3506 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:30.859108 3506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:30.859465 3506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:30.863242 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "72bfd6559c554c69b88a50644afc95e7"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "9f97d958ce2dc5727b3ebd6df0cc993d"
server_key_iv: "c4f8733a9869fa0e1c8636fa0b7da6cb"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.864336 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "72bfd6559c554c69b88a50644afc95e7"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "9f97d958ce2dc5727b3ebd6df0cc993d"
server_key_iv: "c4f8733a9869fa0e1c8636fa0b7da6cb"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.870847 3506 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 11:47:30.874979 3523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:30.876567 3506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260810 11:47:30.876791 3506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "72bfd6559c554c69b88a50644afc95e7"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "9f97d958ce2dc5727b3ebd6df0cc993d"
server_key_iv: "c4f8733a9869fa0e1c8636fa0b7da6cb"
server_key_version: "encryptionkey@0"
I20260810 11:47:30.877082 3506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:30.895256 3506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:30.900233 3506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:30.900583 3506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:30.901738 3506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:30.903255 3506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:30.903344 3506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:30.903544 3506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:30.903609 3506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:30.938086 3506 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:46065
I20260810 11:47:30.938156 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:46065 every 8 connection(s)
I20260810 11:47:30.939585 3506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:30.942579 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3506
I20260810 11:47:30.942864 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:47:30.943774 20854 external_mini_cluster.cc:1428] Setting key b5bdf372e407ef5851149747dae6b317
Aug 10 11:47:30 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362450, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:30.965163 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:30.942805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:47491 (local address 127.20.93.190:42899)
0810 11:47:30.943399 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:30.943404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:30.944892 (+ 1488us) server_negotiation.cc:407] Connection header received
0810 11:47:30.947542 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:30.947551 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:30.947663 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:30.947792 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:30.949463 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.950720 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.952475 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.953259 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.953424 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:30.957983 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:30.957993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:30.958001 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:30.958064 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:30.961230 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:30.962385 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:30.962390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:30.962393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:30.962533 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:30.963104 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:30.963107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:30.963109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:30.963419 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:30.963536 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:30.964676 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0810 11:47:30.964890 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":342,"threads_started":1}
I20260810 11:47:30.967026 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:30.943569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:47491)
0810 11:47:30.944625 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:30.944679 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:30.947184 (+ 2505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:30.948029 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:30.948039 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:30.948618 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:30.949247 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:30.949278 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.950920 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.950924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:30.952284 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:30.952300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:30.953506 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:30.953511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:30.953827 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:30.954620 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:30.954645 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:30.957676 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:30.961419 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:30.961433 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:30.961457 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:30.962236 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:30.962653 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:30.962661 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:30.962667 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:30.962976 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:30.963565 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:30.963576 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:30.964128 (+ 552us) client_negotiation.cc:769] Sending connection context
0810 11:47:30.964479 (+ 351us) client_negotiation.cc:240] Negotiation successful
0810 11:47:30.965207 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":379,"threads_started":1}
I20260810 11:47:30.969002 3654 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:30.969396 3654 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:30.970322 3654 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:30.972791 3437 ts_manager.cc:194] Registered new tserver with Master: 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065)
I20260810 11:47:30.974453 3437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:47491
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:47:31.017556 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:31.401875 3662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:31.402192 3662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:31.402395 3662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:31.410190 3662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:31.410323 3662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:31.410526 3662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:31.420794 3662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:31.422343 3662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:31.423945 3662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:31.438207 3671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:31.438179 3668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:31.438145 3669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:31.439049 3662 server_base.cc:1034] running on GCE node
I20260810 11:47:31.439679 3662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:31.440683 3662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:31.441896 3662 hybrid_clock.cc:648] HybridClock initialized: now 1786362451441854 us; error 41 us; skew 500 ppm
I20260810 11:47:31.442377 3662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:31 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:31.445904 3662 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:31.447440 3662 webserver.cc:460] Webserver started at http://127.20.93.130:43277/ using document root <none> and password file <none>
I20260810 11:47:31.448171 3662 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:31.448338 3662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:31.448666 3662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:31.451906 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "6497a6c9c19e4f4894e178ff338eec6c"
format_stamp: "Formatted at 2026-08-10 11:47:31 on dist-test-slave-njxd"
server_key: "54df8e0058aa859a2491d136318c0b15"
server_key_iv: "cb283ea59be121a1d2cf953c5f22a2f7"
server_key_version: "encryptionkey@0"
I20260810 11:47:31.452790 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "6497a6c9c19e4f4894e178ff338eec6c"
format_stamp: "Formatted at 2026-08-10 11:47:31 on dist-test-slave-njxd"
server_key: "54df8e0058aa859a2491d136318c0b15"
server_key_iv: "cb283ea59be121a1d2cf953c5f22a2f7"
server_key_version: "encryptionkey@0"
I20260810 11:47:31.458652 3662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:47:31.462858 3678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:31.464349 3662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:47:31.464550 3662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "6497a6c9c19e4f4894e178ff338eec6c"
format_stamp: "Formatted at 2026-08-10 11:47:31 on dist-test-slave-njxd"
server_key: "54df8e0058aa859a2491d136318c0b15"
server_key_iv: "cb283ea59be121a1d2cf953c5f22a2f7"
server_key_version: "encryptionkey@0"
I20260810 11:47:31.464782 3662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:31.488601 3662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:31.492830 3662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:31.493134 3662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:31.494247 3662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:31.495678 3662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:31.495786 3662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:31.495939 3662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:31.496026 3662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:31.531030 3662 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42965
I20260810 11:47:31.531342 3800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42965 every 8 connection(s)
I20260810 11:47:31.532481 3662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:31.540334 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3662
I20260810 11:47:31.540558 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:47:31.541072 20854 external_mini_cluster.cc:1428] Setting key 7ef5a42a7280afb00ebbfb1c1ba6213f
Aug 10 11:47:31 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:31.555424 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:31.535394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:57949 (local address 127.20.93.190:42899)
0810 11:47:31.535903 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:31.535910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:31.537055 (+ 1145us) server_negotiation.cc:407] Connection header received
0810 11:47:31.539050 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:31.539055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:31.539137 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:31.539247 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:31.541361 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:31.542398 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:31.544121 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:31.544563 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:31.544725 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:31.548515 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:31.548520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:31.548523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:31.548561 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:31.551889 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:31.552807 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:31.552812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:31.552816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:31.552963 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:31.553443 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:31.553448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:31.553451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:31.553810 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:31.553989 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:31.554778 (+ 789us) server_negotiation.cc:299] Negotiation successful
0810 11:47:31.555029 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":237,"threads_started":1}
I20260810 11:47:31.556592 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:31.536050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:57949)
0810 11:47:31.536864 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:31.536904 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:31.538755 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:31.539468 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:31.539476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:31.540257 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:31.541091 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:31.541113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:31.542577 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:31.542590 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:31.543953 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:31.543967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:31.544854 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:31.544862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:31.545107 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:31.545907 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:31.545931 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:31.548213 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:31.552132 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:31.552140 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:31.552156 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:31.552663 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:31.553095 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:31.553100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:31.553103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:31.553318 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:31.553963 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:31.553972 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:31.554365 (+ 393us) client_negotiation.cc:769] Sending connection context
0810 11:47:31.554664 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 11:47:31.555284 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":305,"threads_started":1}
I20260810 11:47:31.558404 3801 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:31.558763 3801 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:31.559674 3801 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:31.561463 3437 ts_manager.cc:194] Registered new tserver with Master: 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
I20260810 11:47:31.562868 3437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:57949
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:47:31.610189 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:31.975365 3807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:31.975731 3807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:31.975835 3807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 11:47:31.978312 3654 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
W20260810 11:47:31.984398 3807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:31.984521 3807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:31.984639 3807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:31.995160 3807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:31.996603 3807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:31.998080 3807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:32.012238 3812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:32.012317 3813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:32.012270 3815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:32.014302 3807 server_base.cc:1034] running on GCE node
I20260810 11:47:32.015043 3807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:32.016055 3807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:32.017298 3807 hybrid_clock.cc:648] HybridClock initialized: now 1786362452017247 us; error 46 us; skew 500 ppm
I20260810 11:47:32.017731 3807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:32 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:32.021170 3807 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:32.022950 3807 webserver.cc:460] Webserver started at http://127.20.93.131:34975/ using document root <none> and password file <none>
I20260810 11:47:32.023766 3807 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:32.023921 3807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:32.024248 3807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:32.027382 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "427852e89adc4b0482187764f5309331"
format_stamp: "Formatted at 2026-08-10 11:47:32 on dist-test-slave-njxd"
server_key: "2fa024e99843c1020bf51799097d3c32"
server_key_iv: "ce759ee2e76f512b887849ffc61b0867"
server_key_version: "encryptionkey@0"
I20260810 11:47:32.028221 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "427852e89adc4b0482187764f5309331"
format_stamp: "Formatted at 2026-08-10 11:47:32 on dist-test-slave-njxd"
server_key: "2fa024e99843c1020bf51799097d3c32"
server_key_iv: "ce759ee2e76f512b887849ffc61b0867"
server_key_version: "encryptionkey@0"
I20260810 11:47:32.034155 3807 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:47:32.038098 3822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:32.039608 3807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:47:32.039824 3807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "427852e89adc4b0482187764f5309331"
format_stamp: "Formatted at 2026-08-10 11:47:32 on dist-test-slave-njxd"
server_key: "2fa024e99843c1020bf51799097d3c32"
server_key_iv: "ce759ee2e76f512b887849ffc61b0867"
server_key_version: "encryptionkey@0"
I20260810 11:47:32.040035 3807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:32.056120 3807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:32.060745 3807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:32.061045 3807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:32.062160 3807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:32.063534 3807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:32.063608 3807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:32.063783 3807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:32.063838 3807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:32.099896 3807 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41023
I20260810 11:47:32.099937 3947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41023 every 8 connection(s)
I20260810 11:47:32.101298 3807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:32.101876 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 3807
I20260810 11:47:32.102125 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:47:32.102810 20854 external_mini_cluster.cc:1428] Setting key 058a0ec3b269eb2821df3db323571618
Aug 10 11:47:32 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:32.123446 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.104930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:39801 (local address 127.20.93.190:42899)
0810 11:47:32.105415 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.105422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.106582 (+ 1160us) server_negotiation.cc:407] Connection header received
0810 11:47:32.108375 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.108379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.108459 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.108553 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:32.110097 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.110910 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.112444 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.112827 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.112922 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.116586 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:32.116591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:32.116594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:32.116630 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:32.120332 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.121169 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.121174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.121178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.121309 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.121835 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.121839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.121841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.122132 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:32.122260 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.122993 (+ 733us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.123178 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":242,"threads_started":1}
I20260810 11:47:32.124423 3952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.105242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:39801)
0810 11:47:32.106404 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.106458 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.108123 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.108735 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.108743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.109272 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:32.109941 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.109960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.111048 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.111052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.112319 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.112331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.113018 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.113022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.113237 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.113991 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:32.114014 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:32.116331 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:32.120499 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:32.120507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:32.120522 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:32.121020 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:32.121411 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:32.121414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:32.121416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:32.121720 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:32.122246 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:32.122252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:32.122599 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.122852 (+ 253us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.123278 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":371,"threads_started":1}
I20260810 11:47:32.126113 3948 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:32.126454 3948 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:32.127250 3948 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:32.128731 3437 ts_manager.cc:194] Registered new tserver with Master: 427852e89adc4b0482187764f5309331 (127.20.93.131:41023)
I20260810 11:47:32.129554 3437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:39801
I20260810 11:47:32.132285 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:47:32.152271 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.137456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.20.93.190:42899)
0810 11:47:32.137687 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.137692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.138120 (+ 428us) server_negotiation.cc:407] Connection header received
0810 11:47:32.138310 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.138313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.138386 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.138481 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:32.139985 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.141019 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.142662 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.143095 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.143209 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.146153 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:32.146158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:32.146162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:32.146202 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:32.149323 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.150261 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.150265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.150268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.150420 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.150845 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.150849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.150851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.151205 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:32.151405 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.151821 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.152021 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:47:32.156755 3437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:47:32.164485 3437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:47:32.199911 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.178309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.20.93.130:42965)
0810 11:47:32.190896 (+ 12587us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.190907 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.190947 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:47:32.191073 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.191083 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.191406 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.192523 (+ 1117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.192767 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.194592 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.196988 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.198571 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.198728 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.198962 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.199394 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.199584 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12423,"thread_start_us":12315,"threads_started":1}
I20260810 11:47:32.200003 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.178295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42965 (local address 127.0.0.1:45232)
0810 11:47:32.189940 (+ 11645us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.189966 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.190104 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.191705 (+ 1601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.191715 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.191748 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.192238 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.192257 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.194770 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.194780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.196767 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.196784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.198830 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.198837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.199107 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.199171 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.199336 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.199558 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11472,"mutex_wait_us":1,"thread_start_us":11913,"threads_started":1}
I20260810 11:47:32.201931 3963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.178264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45896)
0810 11:47:32.188848 (+ 10584us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.188921 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.189188 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.194685 (+ 5497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.194690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.194716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.195221 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.195237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.197193 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.197199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.198876 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.198888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.200913 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.200923 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.201141 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.201204 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.201290 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.201435 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10191,"mutex_wait_us":5,"thread_start_us":10067,"threads_started":1}
I20260810 11:47:32.202661 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.181306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41023 (local address 127.0.0.1:48110)
0810 11:47:32.191321 (+ 10015us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.191339 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.191474 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.192684 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.192688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.192721 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.193213 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.193227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.195364 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.195369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.199076 (+ 3707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.199092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.201772 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.201780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.201991 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.202037 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.202140 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.202278 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9887,"mutex_wait_us":166,"thread_start_us":9710,"threads_started":1}
I20260810 11:47:32.202965 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.178944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.20.93.129:46065)
0810 11:47:32.194036 (+ 15092us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.194044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.194093 (+ 49us) server_negotiation.cc:407] Connection header received
0810 11:47:32.194175 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.194181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.194440 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.194683 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.195476 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.196996 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.199066 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.200610 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.202079 (+ 1469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.202323 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.202400 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.202592 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14887,"thread_start_us":14596,"threads_started":1}
I20260810 11:47:32.203235 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.182060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.20.93.131:41023)
0810 11:47:32.188469 (+ 6409us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.188480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.191633 (+ 3153us) server_negotiation.cc:407] Connection header received
0810 11:47:32.191759 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.191767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.192042 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.192362 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.193554 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.195226 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.199254 (+ 4028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.201495 (+ 2241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.201697 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.201955 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.202513 (+ 558us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.202693 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6182,"thread_start_us":5965,"threads_started":1}
I20260810 11:47:32.206060 3730 tablet_service.cc:1467] Processing CreateTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a (DEFAULT_TABLE table=test-table [id=35b943a5bf4445549c6f558658ca32fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:32.206496 3577 tablet_service.cc:1467] Processing CreateTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a (DEFAULT_TABLE table=test-table [id=35b943a5bf4445549c6f558658ca32fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:32.206846 3878 tablet_service.cc:1467] Processing CreateTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a (DEFAULT_TABLE table=test-table [id=35b943a5bf4445549c6f558658ca32fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:32.207770 3730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8654e52d976645a8bfd5c0a4aa84659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:32.207991 3577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8654e52d976645a8bfd5c0a4aa84659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:32.208212 3878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8654e52d976645a8bfd5c0a4aa84659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:32.222018 3968 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Bootstrap starting.
I20260810 11:47:32.224800 3971 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Bootstrap starting.
I20260810 11:47:32.226040 3968 tablet_bootstrap.cc:654] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:32.227139 3969 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Bootstrap starting.
I20260810 11:47:32.228003 3968 log.cc:824] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:32.229174 3971 tablet_bootstrap.cc:654] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:32.230608 3971 log.cc:824] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:32.230953 3969 tablet_bootstrap.cc:654] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:32.232223 3968 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: No bootstrap required, opened a new log
I20260810 11:47:32.232232 3969 log.cc:824] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:32.232712 3968 ts_tablet_manager.cc:1397] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260810 11:47:32.240438 3971 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: No bootstrap required, opened a new log
I20260810 11:47:32.240774 3971 ts_tablet_manager.cc:1397] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260810 11:47:32.241778 3969 tablet_bootstrap.cc:492] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: No bootstrap required, opened a new log
I20260810 11:47:32.242069 3969 ts_tablet_manager.cc:1397] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260810 11:47:32.245448 3968 raft_consensus.cc:348] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.246155 3968 raft_consensus.cc:374] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:32.246311 3968 raft_consensus.cc:729] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72bfd6559c554c69b88a50644afc95e7, State: Initialized, Role: FOLLOWER
I20260810 11:47:32.247184 3968 consensus_queue.cc:260] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.253304 3969 raft_consensus.cc:348] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.253897 3969 raft_consensus.cc:374] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:32.253995 3969 raft_consensus.cc:729] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6497a6c9c19e4f4894e178ff338eec6c, State: Initialized, Role: FOLLOWER
I20260810 11:47:32.254375 3971 raft_consensus.cc:348] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.254998 3971 raft_consensus.cc:374] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:32.254858 3969 consensus_queue.cc:260] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.255131 3971 raft_consensus.cc:729] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 427852e89adc4b0482187764f5309331, State: Initialized, Role: FOLLOWER
I20260810 11:47:32.256036 3971 consensus_queue.cc:260] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.258270 3968 ts_tablet_manager.cc:1428] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Time spent starting tablet: real 0.025s user 0.007s sys 0.016s
I20260810 11:47:32.262588 3948 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
I20260810 11:47:32.263262 3969 ts_tablet_manager.cc:1428] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Time spent starting tablet: real 0.021s user 0.016s sys 0.003s
I20260810 11:47:32.263640 3801 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
I20260810 11:47:32.263830 3971 ts_tablet_manager.cc:1428] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Time spent starting tablet: real 0.023s user 0.013s sys 0.009s
I20260810 11:47:32.274312 3976 raft_consensus.cc:482] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:32.274724 3976 raft_consensus.cc:504] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.276670 3976 leader_election.cc:288] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065), 427852e89adc4b0482187764f5309331 (127.20.93.131:41023)
I20260810 11:47:32.282675 3977 raft_consensus.cc:482] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:32.283026 3977 raft_consensus.cc:504] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.284467 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.276993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:40897 (local address 127.20.93.129:46065)
0810 11:47:32.277256 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.277265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.278131 (+ 866us) server_negotiation.cc:407] Connection header received
0810 11:47:32.278364 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.278369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.278431 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.278528 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.279162 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.279892 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.282075 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.283355 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.283544 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.283626 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.284040 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.284186 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 11:47:32.284714 3802 tablet.cc:2369] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:32.284781 3978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.276890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.20.93.130:40897)
0810 11:47:32.278004 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.278022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.278209 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.278543 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.278548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.278573 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.278997 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.279011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.280030 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.280036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.281887 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.281905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.283585 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.283643 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.283837 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.283876 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.283989 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.284150 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":84,"thread_start_us":718,"threads_started":1}
I20260810 11:47:32.285518 3977 leader_election.cc:288] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965), 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065)
I20260810 11:47:32.286018 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.277156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:34245 (local address 127.20.93.131:41023)
0810 11:47:32.277380 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.277388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.278180 (+ 792us) server_negotiation.cc:407] Connection header received
0810 11:47:32.278378 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.278383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.278455 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.278556 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.279217 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.279959 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.282136 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.283606 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.283751 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.283800 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.284356 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.285741 (+ 1385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:47:32.285941 3605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "6497a6c9c19e4f4894e178ff338eec6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72bfd6559c554c69b88a50644afc95e7" is_pre_election: true
I20260810 11:47:32.286516 3605 raft_consensus.cc:2454] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6497a6c9c19e4f4894e178ff338eec6c in term 0.
I20260810 11:47:32.286754 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.277116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41023 (local address 127.20.93.130:34245)
0810 11:47:32.278060 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.278080 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.278236 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.278570 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.278574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.278596 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.279055 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.279065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.280108 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.280115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.281965 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.281978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.283941 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.283946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.284154 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.284195 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.286196 (+ 2001us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.286350 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":22,"thread_start_us":607,"threads_started":1}
I20260810 11:47:32.287384 3683 leader_election.cc:302] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6497a6c9c19e4f4894e178ff338eec6c, 72bfd6559c554c69b88a50644afc95e7; no voters:
I20260810 11:47:32.287812 3976 raft_consensus.cc:2781] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:47:32.287981 3976 raft_consensus.cc:482] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:47:32.288102 3976 raft_consensus.cc:3037] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:32.287999 3899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "6497a6c9c19e4f4894e178ff338eec6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "427852e89adc4b0482187764f5309331" is_pre_election: true
I20260810 11:47:32.288441 3899 raft_consensus.cc:2454] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6497a6c9c19e4f4894e178ff338eec6c in term 0.
I20260810 11:47:32.291429 3976 raft_consensus.cc:504] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.292115 3976 leader_election.cc:288] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [CANDIDATE]: Term 1 election: Requested vote from peers 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065), 427852e89adc4b0482187764f5309331 (127.20.93.131:41023)
I20260810 11:47:32.292585 3605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "6497a6c9c19e4f4894e178ff338eec6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72bfd6559c554c69b88a50644afc95e7"
I20260810 11:47:32.292722 3899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "6497a6c9c19e4f4894e178ff338eec6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "427852e89adc4b0482187764f5309331"
I20260810 11:47:32.292819 3605 raft_consensus.cc:3037] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:32.292953 3899 raft_consensus.cc:3037] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:32.296377 3605 raft_consensus.cc:2454] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6497a6c9c19e4f4894e178ff338eec6c in term 1.
I20260810 11:47:32.296965 3683 leader_election.cc:302] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6497a6c9c19e4f4894e178ff338eec6c, 72bfd6559c554c69b88a50644afc95e7; no voters:
I20260810 11:47:32.297333 3976 raft_consensus.cc:2781] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:32.297662 3952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.285733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42965 (local address 127.20.93.131:43293)
0810 11:47:32.285901 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.285922 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.286037 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.287480 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.287485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.289837 (+ 2352us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x61200002c928. stack: 00007fa88704b66e 00007fa88704b430 00007fa8860d25ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fa8870410fd 00007fa887040d1e 00007fa899f4a3cd 00007fa899f4a36c 00007fa899f16b3b 00007fa88da528e9 00007fa88da4ee45 00007fa88dad6dca 00007fa88dad4f68 00007fa88db0882f
0810 11:47:32.289859 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.290002 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.290011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.291422 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.291426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.293427 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.293449 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.296122 (+ 2673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.296130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.296428 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.296552 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.297022 (+ 470us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.297194 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"spinlock_wait_cycles":3056512}
I20260810 11:47:32.297761 3899 raft_consensus.cc:2454] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6497a6c9c19e4f4894e178ff338eec6c in term 1.
I20260810 11:47:32.298203 3976 raft_consensus.cc:686] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 1 LEADER]: Becoming Leader. State: Replica: 6497a6c9c19e4f4894e178ff338eec6c, State: Running, Role: LEADER
I20260810 11:47:32.298223 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.286990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:43293 (local address 127.20.93.130:42965)
0810 11:47:32.287164 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.287170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.287191 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:47:32.287254 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.287259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.287333 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.287438 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.290171 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.291250 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.293669 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.294977 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.295147 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.295239 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.296756 (+ 1517us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.296917 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 11:47:32.298877 3976 consensus_queue.cc:237] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } }
I20260810 11:47:32.299532 3751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "427852e89adc4b0482187764f5309331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6497a6c9c19e4f4894e178ff338eec6c" is_pre_election: true
I20260810 11:47:32.303987 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.286710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:33107 (local address 127.20.93.129:46065)
0810 11:47:32.292316 (+ 5606us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.292323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.292343 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:47:32.292406 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.292411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.292578 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.292677 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:32.293856 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.295132 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.301905 (+ 6773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.302883 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.303137 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.303221 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.303548 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.303690 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902}
I20260810 11:47:32.304049 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.286188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.20.93.131:33107)
0810 11:47:32.286761 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:32.286780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:32.288896 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:32.292963 (+ 4067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:32.292971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:32.293004 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:32.293511 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.293528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.295349 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.295356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.301673 (+ 6317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:32.301694 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.303152 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.303158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:32.303345 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.303373 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 11:47:32.303499 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 11:47:32.303659 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":1252,"threads_started":1}
I20260810 11:47:32.304690 3605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8654e52d976645a8bfd5c0a4aa84659a" candidate_uuid: "427852e89adc4b0482187764f5309331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72bfd6559c554c69b88a50644afc95e7" is_pre_election: true
I20260810 11:47:32.305076 3605 raft_consensus.cc:2379] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 427852e89adc4b0482187764f5309331 in current term 1: Already voted for candidate 6497a6c9c19e4f4894e178ff338eec6c in this term.
I20260810 11:47:32.305863 3827 leader_election.cc:302] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 427852e89adc4b0482187764f5309331; no voters: 6497a6c9c19e4f4894e178ff338eec6c, 72bfd6559c554c69b88a50644afc95e7
I20260810 11:47:32.305254 3436 catalog_manager.cc:5495] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130). New cstate: current_term: 1 leader_uuid: "6497a6c9c19e4f4894e178ff338eec6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } health_report { overall_health: UNKNOWN } } }
I20260810 11:47:32.306572 3977 raft_consensus.cc:2726] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 11:47:32.336040 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.329712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.20.93.130:42965)
0810 11:47:32.329920 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.329925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.329944 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:47:32.330152 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.330155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.330239 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.330389 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:47:32.330734 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.331544 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.333374 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.334001 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.334113 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.334725 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:47:32.335390 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:47:32.335515 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.335672 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.335779 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:47:32.344911 3605 raft_consensus.cc:1261] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Refusing update from remote peer 6497a6c9c19e4f4894e178ff338eec6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:32.344975 3899 raft_consensus.cc:1261] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Refusing update from remote peer 6497a6c9c19e4f4894e178ff338eec6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:32.346014 3981 consensus_queue.cc:1035] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:32.346704 3976 consensus_queue.cc:1035] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:47:32.353274 3949 tablet.cc:2369] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:32.360689 3984 mvcc.cc:205] Tried to move back new op lower bound from 7316940604794060800 to 7316940604621447168. Current Snapshot: MvccSnapshot[applied={T|T < 7316940604794060800}]
I20260810 11:47:32.363049 3986 mvcc.cc:205] Tried to move back new op lower bound from 7316940604794060800 to 7316940604621447168. Current Snapshot: MvccSnapshot[applied={T|T < 7316940604794060800}]
I20260810 11:47:32.376996 3436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:32.377367 3436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:32.381842 3436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Sending DeleteTablet for 3 replicas of tablet 8654e52d976645a8bfd5c0a4aa84659a
I20260810 11:47:32.383296 3577 tablet_service.cc:1514] Processing DeleteTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:32 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45896
I20260810 11:47:32.383319 3730 tablet_service.cc:1514] Processing DeleteTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:32 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45232
I20260810 11:47:32.383641 3878 tablet_service.cc:1514] Processing DeleteTablet for tablet 8654e52d976645a8bfd5c0a4aa84659a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:32 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:48110
I20260810 11:47:32.384347 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3506
I20260810 11:47:32.384799 3995 tablet_replica.cc:330] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: stopping tablet replica
I20260810 11:47:32.384907 3993 tablet_replica.cc:330] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: stopping tablet replica
I20260810 11:47:32.385408 3993 raft_consensus.cc:2229] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:32.385413 3995 raft_consensus.cc:2229] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:32.386059 3993 raft_consensus.cc:2258] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:32.386255 3995 raft_consensus.cc:2258] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:32.389174 3995 ts_tablet_manager.cc:1905] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:32.391250 3993 ts_tablet_manager.cc:1905] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:32.398772 3995 ts_tablet_manager.cc:1918] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:32.399101 3995 log.cc:1196] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/8654e52d976645a8bfd5c0a4aa84659a
Aug 10 11:47:32 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:47:32.399994 3995 ts_tablet_manager.cc:1939] T 8654e52d976645a8bfd5c0a4aa84659a P 427852e89adc4b0482187764f5309331: Deleting consensus metadata
I20260810 11:47:32.402580 3421 catalog_manager.cc:4841] TS 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): tablet 8654e52d976645a8bfd5c0a4aa84659a (table test-table [id=35b943a5bf4445549c6f558658ca32fd]) successfully deleted
I20260810 11:47:32.406277 3994 tablet_replica.cc:330] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: stopping tablet replica
I20260810 11:47:32.407220 3994 raft_consensus.cc:2229] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:47:32.407987 3994 raft_consensus.cc:2258] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:32.408553 3993 ts_tablet_manager.cc:1918] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:32.408759 3993 log.cc:1196] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/8654e52d976645a8bfd5c0a4aa84659a
I20260810 11:47:32.409451 3993 ts_tablet_manager.cc:1939] T 8654e52d976645a8bfd5c0a4aa84659a P 72bfd6559c554c69b88a50644afc95e7: Deleting consensus metadata
I20260810 11:47:32.410893 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.385898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.20.93.129:46065)
0810 11:47:32.386115 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.386123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.387381 (+ 1258us) server_negotiation.cc:407] Connection header received
0810 11:47:32.387733 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.387739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.387837 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.387920 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:32.390154 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.391276 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.393634 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.394356 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.394468 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.400971 (+ 6503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:32.400991 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:32.401012 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:32.401066 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:32.406344 (+ 5278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.407379 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.407391 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.407397 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.407599 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.408072 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.408080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.408085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.408643 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:32.409027 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.410315 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.410568 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:47:32.411718 3423 catalog_manager.cc:4841] TS 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065): tablet 8654e52d976645a8bfd5c0a4aa84659a (table test-table [id=35b943a5bf4445549c6f558658ca32fd]) successfully deleted
I20260810 11:47:32.412220 3649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:32.413023 3994 ts_tablet_manager.cc:1905] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:32.421413 3994 ts_tablet_manager.cc:1918] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:32.421705 3994 log.cc:1196] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/8654e52d976645a8bfd5c0a4aa84659a
I20260810 11:47:32.422324 3994 ts_tablet_manager.cc:1939] T 8654e52d976645a8bfd5c0a4aa84659a P 6497a6c9c19e4f4894e178ff338eec6c: Deleting consensus metadata
I20260810 11:47:32.424472 3420 catalog_manager.cc:4841] TS 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): tablet 8654e52d976645a8bfd5c0a4aa84659a (table test-table [id=35b943a5bf4445549c6f558658ca32fd]) successfully deleted
I20260810 11:47:32.508643 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3506
I20260810 11:47:32.538548 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3662
Aug 10 11:47:32 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:47:32.560570 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.539454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.20.93.130:42965)
0810 11:47:32.539718 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.539726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.539749 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:47:32.539990 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.539996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.540094 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.540226 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:32.542204 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.543446 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.545614 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.546347 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.546502 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.552588 (+ 6086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:32.552603 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:32.552636 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:32.552707 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:32.557207 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.558295 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.558304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.558307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.558516 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.558973 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.558976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.558979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.559491 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:32.559767 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.560097 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.560292 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 11:47:32.561404 3795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:32.679654 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3662
I20260810 11:47:32.701713 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3807
Aug 10 11:47:32 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:47:32.717842 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:32.702528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.20.93.131:41023)
0810 11:47:32.702775 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:32.702780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:32.702817 (+ 37us) server_negotiation.cc:407] Connection header received
0810 11:47:32.703004 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:32.703009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:32.703106 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:32.703195 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:32.704581 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.705315 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.706770 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:32.707185 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:32.707308 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:32.711908 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:32.711925 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:32.711942 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:32.711987 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:32.714987 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.715715 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.715723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.715725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.715871 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:32.716206 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:32.716209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:32.716211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:32.716622 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:32.716848 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:32.717124 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 11:47:32.717334 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 11:47:32.718629 3942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:32.814438 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3807
I20260810 11:47:32.833751 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 3400
I20260810 11:47:32.835254 3470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:32.952605 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 3400
I20260810 11:47:32.980496 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--webserver_interface=127.20.93.190
--webserver_port=43073
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:42899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:33.355479 4001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:33.355820 4001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:33.355993 4001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:33.363659 4001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:33.363792 4001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:33.363885 4001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:33.363996 4001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:33.364085 4001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:33.374646 4001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:42899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=43073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:33.376124 4001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:33.377549 4001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:33.406190 4007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:33.406258 4008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:33.406229 4010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:33.407202 4001 server_base.cc:1034] running on GCE node
I20260810 11:47:33.408188 4001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:33.410012 4001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:33.411249 4001 hybrid_clock.cc:648] HybridClock initialized: now 1786362453411175 us; error 51 us; skew 500 ppm
I20260810 11:47:33.411695 4001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:33 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:33.424098 4001 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:33.426389 4001 webserver.cc:460] Webserver started at http://127.20.93.190:43073/ using document root <none> and password file <none>
I20260810 11:47:33.427362 4001 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:33.427582 4001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:33.435118 4001 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:47:33.439061 4018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:33.440892 4001 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:47:33.441134 4001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "f4dd5b63ad984fbcba214cb5b2647db8"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "fb2775441450868f5c0c729102951c07"
server_key_iv: "afaa6497c1145e3306e136164e8ce88e"
server_key_version: "encryptionkey@0"
I20260810 11:47:33.442476 4001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:33.468837 4001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:33.474850 4001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:33.475229 4001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:33.497802 4001 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:42899
I20260810 11:47:33.497850 4073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:42899 every 8 connection(s)
I20260810 11:47:33.499591 4001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:33.503445 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4001
I20260810 11:47:33.504341 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:46065
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=40743
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:33.507782 4077 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:47:33.511590 4077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap starting.
I20260810 11:47:33.533318 4077 log.cc:824] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:33.547565 4077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:47:33.548218 4077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap complete.
I20260810 11:47:33.558576 4077 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:33.559523 4077 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Initialized, Role: FOLLOWER
I20260810 11:47:33.560398 4077 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:33.560678 4077 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:33.560809 4077 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:33.561012 4077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:47:33.565966 4077 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:33.566696 4077 leader_election.cc:302] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4dd5b63ad984fbcba214cb5b2647db8; no voters:
I20260810 11:47:33.567435 4077 leader_election.cc:288] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:47:33.567590 4081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:47:33.568914 4081 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 LEADER]: Becoming Leader. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Running, Role: LEADER
I20260810 11:47:33.569718 4081 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:33.570339 4077 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:33.573040 4083 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4dd5b63ad984fbcba214cb5b2647db8. Latest consensus state: current_term: 2 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:33.573412 4082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:33.573765 4083 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:33.573839 4082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:33.585346 4087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:33.597218 4087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:33.599996 4087 catalog_manager.cc:1259] Loaded cluster ID: 208d0898738b4d4184439bf03807b42c
I20260810 11:47:33.600281 4087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:33.605309 4087 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 11:47:33.631760 4087 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 11:47:33.650419 4087 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:33.651925 4087 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Loaded TSK: 0
I20260810 11:47:33.653204 4087 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:47:33.919766 4079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:33.920142 4079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:33.920311 4079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:33.927997 4079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:33.928175 4079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:33.928356 4079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:33.940069 4079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:46065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=40743
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:33.941560 4079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:33.943060 4079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:33.959919 4115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:33.959990 4118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:33.959874 4116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:33.961459 4079 server_base.cc:1034] running on GCE node
I20260810 11:47:33.962157 4079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:33.963358 4079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:33.964596 4079 hybrid_clock.cc:648] HybridClock initialized: now 1786362453964538 us; error 52 us; skew 500 ppm
I20260810 11:47:33.964951 4079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:33 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:33.968572 4079 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:33.970504 4079 webserver.cc:460] Webserver started at http://127.20.93.129:40743/ using document root <none> and password file <none>
I20260810 11:47:33.971318 4079 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:33.971498 4079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:33.978067 4079 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 11:47:33.981369 4125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:33.983124 4079 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:47:33.983352 4079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "72bfd6559c554c69b88a50644afc95e7"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "9f97d958ce2dc5727b3ebd6df0cc993d"
server_key_iv: "c4f8733a9869fa0e1c8636fa0b7da6cb"
server_key_version: "encryptionkey@0"
I20260810 11:47:33.984472 4079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:34.002117 4079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:34.007256 4079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:34.007637 4079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:34.008749 4079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:34.010404 4079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:34.010496 4079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.010651 4079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:34.010742 4079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.046911 4079 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:46065
I20260810 11:47:34.047009 4249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:46065 every 8 connection(s)
I20260810 11:47:34.048661 4079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:34.059108 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4079
I20260810 11:47:34.059988 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:42965
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=43277
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:34 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:34.076443 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:34.051731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:38581 (local address 127.20.93.190:42899)
0810 11:47:34.052336 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:34.052349 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:34.052987 (+ 638us) server_negotiation.cc:407] Connection header received
0810 11:47:34.055272 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:34.055286 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:34.055635 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:34.055974 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:34.057528 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.058991 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.061705 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.062424 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.062585 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:34.067651 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:34.067660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:34.067686 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:34.067730 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:34.071354 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:34.072289 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:34.072296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:34.072303 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:34.072496 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:34.072985 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:34.072989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:34.072991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:34.073497 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:34.073758 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:34.074662 (+ 904us) server_negotiation.cc:299] Negotiation successful
0810 11:47:34.075077 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":280,"threads_started":1}
I20260810 11:47:34.076443 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:34.051931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:38581)
0810 11:47:34.052799 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:34.052848 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:34.054728 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:34.056159 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:34.056167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:34.056722 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:34.057302 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:34.057321 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.059218 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.059223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:34.061446 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:34.061468 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.062714 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.062724 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:34.063048 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:34.063913 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:34.063957 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:34.067291 (+ 3334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:34.071558 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:34.071571 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:34.071592 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:34.072106 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:34.072630 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:34.072636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:34.072641 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:34.072842 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:34.073691 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:34.073700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:34.074075 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 11:47:34.074378 (+ 303us) client_negotiation.cc:240] Negotiation successful
0810 11:47:34.075025 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":364,"threads_started":1}
I20260810 11:47:34.079535 4250 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:34.079917 4250 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:34.080994 4250 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:34.083559 4036 ts_manager.cc:194] Registered new tserver with Master: 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065)
I20260810 11:47:34.085255 4036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:38581
W20260810 11:47:34.425226 4254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:34.425557 4254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:34.425779 4254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:34.433348 4254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:34.433477 4254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:34.433691 4254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:34.444120 4254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:42965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=43277
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:34.445525 4254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:34.447078 4254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:34.461462 4264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:34.461440 4266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:34.461668 4262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:34.462314 4254 server_base.cc:1034] running on GCE node
I20260810 11:47:34.463079 4254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:34.464198 4254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:34.465428 4254 hybrid_clock.cc:648] HybridClock initialized: now 1786362454465337 us; error 73 us; skew 500 ppm
I20260810 11:47:34.465835 4254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:34 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362454, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:34.469199 4254 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:34.470976 4254 webserver.cc:460] Webserver started at http://127.20.93.130:43277/ using document root <none> and password file <none>
I20260810 11:47:34.471858 4254 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:34.472043 4254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:34.478340 4254 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 11:47:34.481763 4273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.483379 4254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:34.483601 4254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "6497a6c9c19e4f4894e178ff338eec6c"
format_stamp: "Formatted at 2026-08-10 11:47:31 on dist-test-slave-njxd"
server_key: "54df8e0058aa859a2491d136318c0b15"
server_key_iv: "cb283ea59be121a1d2cf953c5f22a2f7"
server_key_version: "encryptionkey@0"
I20260810 11:47:34.484690 4254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:34.514966 4254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:34.519980 4254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:34.520310 4254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:34.521622 4254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:34.523321 4254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:34.523454 4254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.523615 4254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:34.523712 4254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.560878 4254 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42965
I20260810 11:47:34.560940 4394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42965 every 8 connection(s)
I20260810 11:47:34.562623 4254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:34.564854 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4254
I20260810 11:47:34.565544 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:41023
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=34975
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:47:34 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362454, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:34.586898 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:34.565787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:58059 (local address 127.20.93.190:42899)
0810 11:47:34.569870 (+ 4083us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:34.569879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:34.569907 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:47:34.569984 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:34.569991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:34.570139 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:34.570263 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:34.572199 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.573349 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.575172 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.575929 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.576085 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:34.580465 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:34.580471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:34.580474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:34.580512 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:34.583344 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:34.584210 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:34.584215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:34.584217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:34.584360 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:34.584866 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:34.584870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:34.584872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:34.585177 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:34.585355 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:34.586261 (+ 906us) server_negotiation.cc:299] Negotiation successful
0810 11:47:34.586540 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3870}
I20260810 11:47:34.588115 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:34.566393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:58059)
0810 11:47:34.567393 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:34.567451 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:34.569266 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:34.570563 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:34.570573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:34.571277 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:34.571989 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:34.572011 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.573546 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.573551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:34.574997 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:34.575010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:34.576177 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:34.576182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:34.576402 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:34.577193 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:34.577231 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:34.580191 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:34.583521 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:34.583531 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:34.583552 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:34.584051 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:34.584485 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:34.584490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:34.584494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:34.584716 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:34.585345 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:34.585353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:34.585821 (+ 468us) client_negotiation.cc:769] Sending connection context
0810 11:47:34.586130 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 11:47:34.586745 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":460,"threads_started":1}
I20260810 11:47:34.589960 4395 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:34.590384 4395 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:34.591532 4395 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:34.593477 4036 ts_manager.cc:194] Registered new tserver with Master: 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
I20260810 11:47:34.594384 4036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:58059
W20260810 11:47:34.930001 4399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:34.930320 4399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:34.930493 4399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:34.938251 4399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:34.938380 4399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:34.938563 4399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:34.948948 4399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:41023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=34975
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:34.950542 4399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:34.952055 4399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:34.966612 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:34.966728 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:34.966702 4409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:34.967963 4399 server_base.cc:1034] running on GCE node
I20260810 11:47:34.968653 4399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:34.969749 4399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:34.970948 4399 hybrid_clock.cc:648] HybridClock initialized: now 1786362454970852 us; error 82 us; skew 500 ppm
I20260810 11:47:34.971342 4399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:34 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362454, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:34.974992 4399 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:34.976559 4399 webserver.cc:460] Webserver started at http://127.20.93.131:34975/ using document root <none> and password file <none>
I20260810 11:47:34.977385 4399 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:34.977525 4399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:34.983568 4399 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:47:34.987108 4416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:34.989001 4399 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:47:34.989261 4399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "427852e89adc4b0482187764f5309331"
format_stamp: "Formatted at 2026-08-10 11:47:32 on dist-test-slave-njxd"
server_key: "2fa024e99843c1020bf51799097d3c32"
server_key_iv: "ce759ee2e76f512b887849ffc61b0867"
server_key_version: "encryptionkey@0"
I20260810 11:47:34.990465 4399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:35.006700 4399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:35.011579 4399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:35.011914 4399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:35.013027 4399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:35.014631 4399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:35.014734 4399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:35.014897 4399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:35.014982 4399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:35.050135 4399 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41023
I20260810 11:47:35.050176 4539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41023 every 8 connection(s)
I20260810 11:47:35.052106 4399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:35.059258 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4399
Aug 10 11:47:35 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362454, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:35.078473 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.055096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:50109 (local address 127.20.93.190:42899)
0810 11:47:35.055323 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.055330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.056695 (+ 1365us) server_negotiation.cc:407] Connection header received
0810 11:47:35.058679 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.058684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.058762 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.058864 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:35.061128 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.062180 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.064296 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.064828 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.064916 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.069659 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:35.069668 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:35.069673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:35.069725 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:35.074008 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.075252 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.075262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.075265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.075482 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.076113 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.076118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.076122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.076569 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:35.076751 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.077816 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.078084 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:47:35.080205 4542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.055668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:50109)
0810 11:47:35.056510 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.056563 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.058356 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.059211 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.059227 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.060011 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:35.060909 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.060940 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.062346 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.062355 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.064153 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.064167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.065064 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.065072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.065402 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.066200 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:35.066236 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:35.069303 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:35.074250 (+ 4947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:35.074262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:35.074285 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:35.075074 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:35.075649 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:35.075656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:35.075659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:35.075941 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:35.076737 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:35.076748 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:35.077268 (+ 520us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.077650 (+ 382us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.078394 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":278,"threads_started":1}
I20260810 11:47:35.081774 4544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.060369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.20.93.190:42899)
0810 11:47:35.060910 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.060916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.061270 (+ 354us) server_negotiation.cc:407] Connection header received
0810 11:47:35.061402 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.061406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.061484 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.061658 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:35.063558 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.064341 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.066319 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.066793 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.066893 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.072059 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:35.072097 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:35.072104 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:35.073912 (+ 1808us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:35.077502 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.079598 (+ 2096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.079603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.079612 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.079763 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.080172 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.080179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.080184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.080579 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:35.080873 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.081121 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.081365 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":268,"threads_started":1}
I20260810 11:47:35.082160 4540 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42899
I20260810 11:47:35.082497 4540 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:35.083361 4540 heartbeater.cc:511] Master 127.20.93.190:42899 requested a full tablet report, sending...
I20260810 11:47:35.084985 4036 ts_manager.cc:194] Registered new tserver with Master: 427852e89adc4b0482187764f5309331 (127.20.93.131:41023)
I20260810 11:47:35.085891 4036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:50109
I20260810 11:47:35.089798 4250 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
I20260810 11:47:35.097324 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:47:35.105465 4544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.098435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.20.93.190:42899)
0810 11:47:35.098705 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.098712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.099033 (+ 321us) server_negotiation.cc:407] Connection header received
0810 11:47:35.099225 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.099229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.099441 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.099591 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:47:35.100215 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.100963 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.102815 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.103326 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.103446 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.104104 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:47:35.104716 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:47:35.104856 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.105009 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.105097 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 11:47:35.106635 4036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:47:35.114310 4036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:47:35.156127 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.127195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42965 (local address 127.0.0.1:45248)
0810 11:47:35.146908 (+ 19713us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.146938 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.147076 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.147917 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.147922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.147950 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:35.148533 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.148551 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.150505 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.150511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.152567 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.152583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.154962 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.154969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.155184 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.155249 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.155402 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.155631 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19547,"mutex_wait_us":97,"thread_start_us":19625,"threads_started":1}
I20260810 11:47:35.156579 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.127257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.20.93.131:41023)
0810 11:47:35.146195 (+ 18938us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.146210 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.146269 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:47:35.146926 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.146938 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.147301 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.147537 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:35.148557 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.150122 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.152992 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.155006 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.155171 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.155407 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.156010 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.156209 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18641,"thread_start_us":19028,"threads_started":1}
I20260810 11:47:35.156816 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.128083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.20.93.130:42965)
0810 11:47:35.136636 (+ 8553us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.136651 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.147154 (+ 10503us) server_negotiation.cc:407] Connection header received
0810 11:47:35.147290 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.147298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.147639 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.147862 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:35.148746 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.150331 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.152767 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.154375 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.154556 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.154785 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.156246 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.156455 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8294,"thread_start_us":8113,"threads_started":1}
I20260810 11:47:35.156664 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.127574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45920)
0810 11:47:35.147136 (+ 19562us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.147159 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.147313 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.149026 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.149031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.149059 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:35.149539 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.149553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.151576 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.151585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.153471 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.153483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.155746 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.155752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.155933 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.155970 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.156091 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.156253 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19410,"mutex_wait_us":1,"thread_start_us":18990,"threads_started":1}
I20260810 11:47:35.157394 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.128278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.20.93.129:46065)
0810 11:47:35.137895 (+ 9617us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.137907 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.148143 (+ 10236us) server_negotiation.cc:407] Connection header received
0810 11:47:35.148296 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.148304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.148605 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.148816 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:35.149778 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.151388 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.153667 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.155494 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.155643 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.155878 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.156771 (+ 893us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.156988 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9387,"thread_start_us":9181,"threads_started":1}
I20260810 11:47:35.157972 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.126884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41023 (local address 127.0.0.1:48126)
0810 11:47:35.145999 (+ 19115us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.146049 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.146296 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.147623 (+ 1327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.147630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.147665 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:35.148273 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.148290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.150294 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.150299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.152795 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.152810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.155474 (+ 2664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.155480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.155677 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.155708 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.155821 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.155980 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18820,"mutex_wait_us":1,"thread_start_us":18582,"threads_started":1}
I20260810 11:47:35.161752 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 108738e6f9674ca58f28c02d1de2769f (DEFAULT_TABLE table=test-table [id=c301e49ba03b4b978e638f6a7a3ec4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:35.161761 4325 tablet_service.cc:1467] Processing CreateTablet for tablet 108738e6f9674ca58f28c02d1de2769f (DEFAULT_TABLE table=test-table [id=c301e49ba03b4b978e638f6a7a3ec4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:35.162142 4470 tablet_service.cc:1467] Processing CreateTablet for tablet 108738e6f9674ca58f28c02d1de2769f (DEFAULT_TABLE table=test-table [id=c301e49ba03b4b978e638f6a7a3ec4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:35.163357 4325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108738e6f9674ca58f28c02d1de2769f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:35.163368 4470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108738e6f9674ca58f28c02d1de2769f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:35.163393 4178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108738e6f9674ca58f28c02d1de2769f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:35.176793 4558 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Bootstrap starting.
I20260810 11:47:35.179623 4559 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Bootstrap starting.
I20260810 11:47:35.181190 4558 tablet_bootstrap.cc:654] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:35.182399 4560 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Bootstrap starting.
I20260810 11:47:35.183723 4559 tablet_bootstrap.cc:654] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:35.184288 4558 log.cc:824] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:35.185073 4559 log.cc:824] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:35.185956 4560 tablet_bootstrap.cc:654] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:35.187165 4560 log.cc:824] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:35.187744 4558 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: No bootstrap required, opened a new log
I20260810 11:47:35.188097 4558 ts_tablet_manager.cc:1397] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 11:47:35.188735 4559 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: No bootstrap required, opened a new log
I20260810 11:47:35.189083 4559 ts_tablet_manager.cc:1397] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260810 11:47:35.190675 4560 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: No bootstrap required, opened a new log
I20260810 11:47:35.190943 4560 ts_tablet_manager.cc:1397] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:47:35.198063 4559 raft_consensus.cc:348] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.198498 4559 raft_consensus.cc:374] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:35.198606 4559 raft_consensus.cc:729] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72bfd6559c554c69b88a50644afc95e7, State: Initialized, Role: FOLLOWER
I20260810 11:47:35.199275 4559 consensus_queue.cc:260] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.199309 4558 raft_consensus.cc:348] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.199779 4558 raft_consensus.cc:374] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:35.199882 4558 raft_consensus.cc:729] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6497a6c9c19e4f4894e178ff338eec6c, State: Initialized, Role: FOLLOWER
I20260810 11:47:35.200492 4558 consensus_queue.cc:260] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.201948 4395 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
I20260810 11:47:35.202512 4560 raft_consensus.cc:348] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.202991 4560 raft_consensus.cc:374] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:35.203086 4560 raft_consensus.cc:729] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 427852e89adc4b0482187764f5309331, State: Initialized, Role: FOLLOWER
I20260810 11:47:35.204443 4558 ts_tablet_manager.cc:1428] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260810 11:47:35.203745 4560 consensus_queue.cc:260] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.207080 4559 ts_tablet_manager.cc:1428] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260810 11:47:35.208889 4540 heartbeater.cc:503] Master 127.20.93.190:42899 was elected leader, sending a full tablet report...
I20260810 11:47:35.209460 4560 ts_tablet_manager.cc:1428] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260810 11:47:35.214603 4564 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:35.214898 4564 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.216717 4564 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
I20260810 11:47:35.224069 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.217128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:41787 (local address 127.20.93.131:41023)
0810 11:47:35.217399 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.217409 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.218312 (+ 903us) server_negotiation.cc:407] Connection header received
0810 11:47:35.218556 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.218561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.218643 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.218743 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:35.219346 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.220121 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.222134 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.223049 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.223249 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.223340 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.223649 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.223789 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:47:35.224220 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.217083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41023 (local address 127.20.93.129:41787)
0810 11:47:35.218192 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.218208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.218405 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.218762 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.218767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.218836 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:35.219200 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.219209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.220261 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.220267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.221966 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.221979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.223307 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.223314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.223474 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.223498 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.223613 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.223757 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":37,"spinlock_wait_cycles":5632,"thread_start_us":736,"threads_started":1}
I20260810 11:47:35.224300 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.217782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:36681 (local address 127.20.93.130:42965)
0810 11:47:35.218034 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.218043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.218737 (+ 694us) server_negotiation.cc:407] Connection header received
0810 11:47:35.218997 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.219001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.219062 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.219163 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:35.219708 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.220418 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.222456 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.223288 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.223490 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.223572 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.223881 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.224015 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:47:35.224723 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.217107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42965 (local address 127.20.93.129:36681)
0810 11:47:35.218616 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.218631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:35.218841 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:35.219188 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:35.219193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:35.219218 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:35.219537 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.219547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.220572 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.220578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.222278 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:35.222297 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.223534 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.223542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:35.223700 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.223724 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:47:35.223840 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 11:47:35.223981 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"mutex_wait_us":5,"spinlock_wait_cycles":45696,"thread_start_us":1151,"threads_started":1}
I20260810 11:47:35.225350 4493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108738e6f9674ca58f28c02d1de2769f" candidate_uuid: "72bfd6559c554c69b88a50644afc95e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "427852e89adc4b0482187764f5309331" is_pre_election: true
I20260810 11:47:35.225792 4348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108738e6f9674ca58f28c02d1de2769f" candidate_uuid: "72bfd6559c554c69b88a50644afc95e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6497a6c9c19e4f4894e178ff338eec6c" is_pre_election: true
I20260810 11:47:35.225915 4493 raft_consensus.cc:2454] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72bfd6559c554c69b88a50644afc95e7 in term 0.
I20260810 11:47:35.226282 4348 raft_consensus.cc:2454] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72bfd6559c554c69b88a50644afc95e7 in term 0.
I20260810 11:47:35.226670 4129 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 427852e89adc4b0482187764f5309331, 72bfd6559c554c69b88a50644afc95e7; no voters:
I20260810 11:47:35.227092 4564 raft_consensus.cc:2781] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:47:35.227254 4564 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:47:35.227361 4564 raft_consensus.cc:3037] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:35.229760 4564 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.230453 4564 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 1 election: Requested vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
I20260810 11:47:35.230871 4493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108738e6f9674ca58f28c02d1de2769f" candidate_uuid: "72bfd6559c554c69b88a50644afc95e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "427852e89adc4b0482187764f5309331"
I20260810 11:47:35.231083 4493 raft_consensus.cc:3037] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:35.231624 4348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108738e6f9674ca58f28c02d1de2769f" candidate_uuid: "72bfd6559c554c69b88a50644afc95e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6497a6c9c19e4f4894e178ff338eec6c"
I20260810 11:47:35.231875 4348 raft_consensus.cc:3037] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:35.233928 4493 raft_consensus.cc:2454] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72bfd6559c554c69b88a50644afc95e7 in term 1.
I20260810 11:47:35.234418 4129 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 427852e89adc4b0482187764f5309331, 72bfd6559c554c69b88a50644afc95e7; no voters:
I20260810 11:47:35.234835 4564 raft_consensus.cc:2781] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:35.234831 4348 raft_consensus.cc:2454] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72bfd6559c554c69b88a50644afc95e7 in term 1.
I20260810 11:47:35.235437 4564 raft_consensus.cc:686] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 LEADER]: Becoming Leader. State: Replica: 72bfd6559c554c69b88a50644afc95e7, State: Running, Role: LEADER
I20260810 11:47:35.235975 4564 consensus_queue.cc:237] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:35.241096 4035 catalog_manager.cc:5495] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "72bfd6559c554c69b88a50644afc95e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } health_report { overall_health: HEALTHY } } }
I20260810 11:47:35.271739 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.265754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.20.93.129:46065)
0810 11:47:35.265965 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.265970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.265988 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:47:35.266100 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.266104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.266188 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.266327 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:47:35.266728 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.267519 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.269202 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.269649 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.269760 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.270372 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:47:35.271085 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:47:35.271227 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.271351 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.271437 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:47:35.280493 4348 raft_consensus.cc:1261] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Refusing update from remote peer 72bfd6559c554c69b88a50644afc95e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:35.280507 4493 raft_consensus.cc:1261] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Refusing update from remote peer 72bfd6559c554c69b88a50644afc95e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:35.281561 4570 consensus_queue.cc:1035] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:35.282004 4564 consensus_queue.cc:1035] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:35.292330 4573 mvcc.cc:205] Tried to move back new op lower bound from 7316940616818692096 to 7316940616650391552. Current Snapshot: MvccSnapshot[applied={T|T < 7316940616818692096}]
I20260810 11:47:35.299952 4574 mvcc.cc:205] Tried to move back new op lower bound from 7316940616818692096 to 7316940616650391552. Current Snapshot: MvccSnapshot[applied={T|T < 7316940616818692096}]
I20260810 11:47:35.316135 4035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:35.316430 4035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:35.320288 4035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Sending DeleteTablet for 3 replicas of tablet 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:35.321769 4325 tablet_service.cc:1514] Processing DeleteTablet for tablet 108738e6f9674ca58f28c02d1de2769f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:35 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45248
I20260810 11:47:35.322096 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 108738e6f9674ca58f28c02d1de2769f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:35 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45920
I20260810 11:47:35.322131 4470 tablet_service.cc:1514] Processing DeleteTablet for tablet 108738e6f9674ca58f28c02d1de2769f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:35 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:48126
I20260810 11:47:35.323474 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4079
I20260810 11:47:35.334290 4586 tablet_replica.cc:330] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: stopping tablet replica
I20260810 11:47:35.334965 4586 raft_consensus.cc:2229] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:35.335719 4586 raft_consensus.cc:2258] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:35.346632 4587 tablet_replica.cc:330] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: stopping tablet replica
I20260810 11:47:35.347157 4587 raft_consensus.cc:2229] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:47:35.347918 4587 raft_consensus.cc:2258] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:35.348232 4585 tablet_replica.cc:330] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: stopping tablet replica
I20260810 11:47:35.348688 4585 raft_consensus.cc:2229] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:35.349267 4585 raft_consensus.cc:2258] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:35.349838 4586 ts_tablet_manager.cc:1905] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:35.351595 4585 ts_tablet_manager.cc:1905] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:35.352958 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.325378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.20.93.129:46065)
0810 11:47:35.325629 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.325637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.325665 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:47:35.325741 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.325750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.325862 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.325984 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:35.328772 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.330485 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.333177 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.335729 (+ 2552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.335927 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.341469 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:35.341498 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:35.341528 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:35.341632 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:35.347782 (+ 6150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.349029 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.349040 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.349044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.349231 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.349779 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.349784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.349788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.350295 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:35.350595 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.352220 (+ 1625us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.352511 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:47:35.353802 4244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:35.358049 4586 ts_tablet_manager.cc:1918] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:35.358242 4586 log.cc:1196] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:35.358857 4586 ts_tablet_manager.cc:1939] T 108738e6f9674ca58f28c02d1de2769f P 427852e89adc4b0482187764f5309331: Deleting consensus metadata
I20260810 11:47:35.360708 4585 ts_tablet_manager.cc:1918] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:35.360910 4585 log.cc:1196] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:35.361337 4021 catalog_manager.cc:4841] TS 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): tablet 108738e6f9674ca58f28c02d1de2769f (table test-table [id=c301e49ba03b4b978e638f6a7a3ec4c5]) successfully deleted
I20260810 11:47:35.361635 4585 ts_tablet_manager.cc:1939] T 108738e6f9674ca58f28c02d1de2769f P 6497a6c9c19e4f4894e178ff338eec6c: Deleting consensus metadata
I20260810 11:47:35.363363 4020 catalog_manager.cc:4841] TS 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): tablet 108738e6f9674ca58f28c02d1de2769f (table test-table [id=c301e49ba03b4b978e638f6a7a3ec4c5]) successfully deleted
I20260810 11:47:35.458112 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4079
I20260810 11:47:35.458230 4587 ts_tablet_manager.cc:1905] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 11:47:35.482227 4022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.93.129:46065 (error 108)
I20260810 11:47:35.482511 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4254
I20260810 11:47:35.484548 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.484004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45952)
0810 11:47:35.484219 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.484355 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 11:47:35.484843 4022 catalog_manager.cc:4584] TS 72bfd6559c554c69b88a50644afc95e7 (127.20.93.129:46065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 108738e6f9674ca58f28c02d1de2769f: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
I20260810 11:47:35.498771 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.483550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.20.93.130:42965)
0810 11:47:35.483735 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.483742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.483862 (+ 120us) server_negotiation.cc:407] Connection header received
0810 11:47:35.484075 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.484081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.484171 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.484291 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:35.486138 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.486931 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.488371 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.488883 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.489004 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.491958 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:35.491975 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:35.491992 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:35.492037 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:35.495374 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.496336 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.496344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.496346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.496489 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.496895 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.496898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.496901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.497468 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:35.497878 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.498140 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.498381 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:47:35.499696 4390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:35.538800 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.538197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45966)
0810 11:47:35.538463 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.538600 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 11:47:35.539983 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.539632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45968)
0810 11:47:35.539777 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.539870 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 11:47:35.601183 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4254
I20260810 11:47:35.601315 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.600277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45970)
0810 11:47:35.600940 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.601059 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500}
I20260810 11:47:35.604434 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.604037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45980)
0810 11:47:35.604226 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.604309 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 11:47:35.623128 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4399
I20260810 11:47:35.638623 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.624123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.20.93.131:41023)
0810 11:47:35.624305 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:35.624309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:35.624329 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:47:35.624417 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:35.624421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:35.624488 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:35.624583 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:35.626294 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.627087 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.628571 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:35.628999 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:35.629107 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:35.632107 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:35.632124 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:35.632141 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:35.632189 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:35.635529 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.636402 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.636411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.636414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.636571 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:35.636965 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:35.636968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:35.636970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:35.637399 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:35.637859 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:35.638152 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 11:47:35.638356 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:47:35.639334 4535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:35.685225 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.684717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45982)
0810 11:47:35.684955 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.685057 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 11:47:35.686561 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.686054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:45996)
0810 11:47:35.686318 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.686415 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 11:47:35.734261 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4399
I20260810 11:47:35.754024 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4001
I20260810 11:47:35.755282 4069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:35.875036 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:35.874390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46065 (local address 127.0.0.1:46012)
0810 11:47:35.874643 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:35.874781 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:47:35.875042 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4001
I20260810 11:47:35.903657 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--webserver_interface=127.20.93.190
--webserver_port=43073
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:42899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:36.279958 4593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:36.280284 4593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:36.280457 4593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:36.288341 4593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:36.288480 4593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:36.288571 4593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:36.288677 4593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:36.288765 4593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:36.299782 4593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:42899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=43073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:36.301357 4593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:36.302970 4593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:36.316306 4598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:36.316215 4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:36.316272 4602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:36.317205 4593 server_base.cc:1034] running on GCE node
I20260810 11:47:36.318176 4593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:36.319854 4593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:36.321111 4593 hybrid_clock.cc:648] HybridClock initialized: now 1786362456321043 us; error 60 us; skew 500 ppm
I20260810 11:47:36.321514 4593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:36 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:36.325040 4593 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:36.326824 4593 webserver.cc:460] Webserver started at http://127.20.93.190:43073/ using document root <none> and password file <none>
I20260810 11:47:36.327580 4593 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:36.327737 4593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:36.333840 4593 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:47:36.337297 4610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:36.338953 4593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:36.339170 4593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "f4dd5b63ad984fbcba214cb5b2647db8"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "fb2775441450868f5c0c729102951c07"
server_key_iv: "afaa6497c1145e3306e136164e8ce88e"
server_key_version: "encryptionkey@0"
I20260810 11:47:36.340260 4593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:36.374585 4593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:36.379240 4593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:36.379559 4593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:36.400616 4593 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:42899
I20260810 11:47:36.400637 4666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:42899 every 8 connection(s)
I20260810 11:47:36.402499 4593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:36.406742 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4593
I20260810 11:47:36.407585 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:46065
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=40743
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:36.410320 4667 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:47:36.413919 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap starting.
I20260810 11:47:36.436276 4667 log.cc:824] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:36.453890 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:47:36.454499 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8: Bootstrap complete.
I20260810 11:47:36.464144 4667 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:36.465183 4667 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Initialized, Role: FOLLOWER
I20260810 11:47:36.465950 4667 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:36.466231 4667 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:36.466368 4667 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:36.466570 4667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 2 FOLLOWER]: Advancing to term 3
I20260810 11:47:36.470922 4667 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:36.471539 4667 leader_election.cc:302] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4dd5b63ad984fbcba214cb5b2647db8; no voters:
I20260810 11:47:36.472304 4667 leader_election.cc:288] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 11:47:36.472451 4671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 11:47:36.473767 4671 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [term 3 LEADER]: Becoming Leader. State: Replica: f4dd5b63ad984fbcba214cb5b2647db8, State: Running, Role: LEADER
I20260810 11:47:36.474435 4671 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } }
I20260810 11:47:36.474817 4667 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:36.477566 4672 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:36.478127 4672 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:36.482951 4673 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4dd5b63ad984fbcba214cb5b2647db8. Latest consensus state: current_term: 3 leader_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4dd5b63ad984fbcba214cb5b2647db8" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42899 } } }
I20260810 11:47:36.483342 4673 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4dd5b63ad984fbcba214cb5b2647db8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:36.489012 4676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:36.500793 4676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:36.503084 4676 catalog_manager.cc:1259] Loaded cluster ID: 208d0898738b4d4184439bf03807b42c
I20260810 11:47:36.503626 4676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 11:47:36.506738 4676 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7936cf09ad google::LogMessage::Fail() at ??:0
@ 0x7f7936cf388a google::LogMessage::SendToLog() at ??:0
@ 0x7f7936cf04de google::LogMessage::Flush() at ??:0
@ 0x7f7936cf1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f794c723512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f794c720976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f794c7a8610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f794c7a8439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f793993c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f793993f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f793993f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f793991aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f793cbf46db start_thread at ??:0
@ 0x7f7935b8a61f clone at ??:0
W20260810 11:47:36.858757 4669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:36.859083 4669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:36.859186 4669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:36.866760 4669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:36.866895 4669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:36.867017 4669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:36.877609 4669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:46065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=40743
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:36.878986 4669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:36.880393 4669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:36.895242 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:36.895339 4716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:36.895242 4713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:36.896188 4669 server_base.cc:1034] running on GCE node
I20260810 11:47:36.896850 4669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:36.897899 4669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:36.899174 4669 hybrid_clock.cc:648] HybridClock initialized: now 1786362456899110 us; error 41 us; skew 500 ppm
I20260810 11:47:36.899570 4669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:36 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:36.903393 4669 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:36.905009 4669 webserver.cc:460] Webserver started at http://127.20.93.129:40743/ using document root <none> and password file <none>
I20260810 11:47:36.905896 4669 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:36.906076 4669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:36.912732 4669 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:47:36.916215 4724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:36.917863 4669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:36.918066 4669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "72bfd6559c554c69b88a50644afc95e7"
format_stamp: "Formatted at 2026-08-10 11:47:30 on dist-test-slave-njxd"
server_key: "9f97d958ce2dc5727b3ebd6df0cc993d"
server_key_iv: "c4f8733a9869fa0e1c8636fa0b7da6cb"
server_key_version: "encryptionkey@0"
I20260810 11:47:36.919157 4669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:36.941087 4669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:36.946281 4669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:36.946609 4669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:36.947752 4669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:36.949905 4732 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 11:47:36.956440 4669 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 11:47:36.956647 4669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260810 11:47:36.956831 4669 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 11:47:36.960354 4669 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 11:47:36.960487 4669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.001s
I20260810 11:47:36.960868 4732 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Bootstrap starting.
I20260810 11:47:36.971175 4732 log.cc:824] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:36.989974 4732 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:47:36.991019 4732 tablet_bootstrap.cc:492] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Bootstrap complete.
I20260810 11:47:36.991901 4732 ts_tablet_manager.cc:1397] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.008s
I20260810 11:47:37.003949 4732 raft_consensus.cc:348] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:37.005353 4732 raft_consensus.cc:729] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72bfd6559c554c69b88a50644afc95e7, State: Initialized, Role: FOLLOWER
I20260810 11:47:37.006332 4669 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:46065
I20260810 11:47:37.006388 4854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:46065 every 8 connection(s)
I20260810 11:47:37.006269 4732 consensus_queue.cc:260] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:37.007864 4669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:37.008392 4732 ts_tablet_manager.cc:1428] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260810 11:47:37.013130 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.011008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:40893)
0810 11:47:37.011853 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.012123 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":345,"threads_started":1}
I20260810 11:47:37.013439 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4669
I20260810 11:47:37.014348 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:42965
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=43277
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:37.014338 4856 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:42899 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:42899: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
I20260810 11:47:37.015375 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.014999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:45965)
0810 11:47:37.015174 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.015263 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 11:47:37.016364 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.015990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:37163)
0810 11:47:37.016152 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.016237 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:47:37.016641 4856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:47:37.393740 4860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:37.394126 4860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:37.394306 4860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:37.402168 4860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:37.402377 4860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:37.402565 4860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:37.413048 4860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:42965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=43277
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:37.414587 4860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:37.416126 4860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:37.431695 4867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:37.431739 4870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:37.431797 4868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:37.433002 4860 server_base.cc:1034] running on GCE node
I20260810 11:47:37.433822 4860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:37.434947 4860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:37.436271 4860 hybrid_clock.cc:648] HybridClock initialized: now 1786362457436220 us; error 46 us; skew 500 ppm
I20260810 11:47:37.436702 4860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:37 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:37.440804 4860 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:37.442703 4860 webserver.cc:460] Webserver started at http://127.20.93.130:43277/ using document root <none> and password file <none>
I20260810 11:47:37.443526 4860 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:37.443701 4860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:37.450096 4860 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:47:37.453775 4877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:37.455654 4860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:47:37.455937 4860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "6497a6c9c19e4f4894e178ff338eec6c"
format_stamp: "Formatted at 2026-08-10 11:47:31 on dist-test-slave-njxd"
server_key: "54df8e0058aa859a2491d136318c0b15"
server_key_iv: "cb283ea59be121a1d2cf953c5f22a2f7"
server_key_version: "encryptionkey@0"
I20260810 11:47:37.457149 4860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:37.482352 4860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:37.487383 4860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:37.487706 4860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:37.488875 4860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:37.490567 4860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:37.490708 4860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:37.490860 4860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:37.490954 4860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:37.531272 4860 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:42965
I20260810 11:47:37.531334 5000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:42965 every 8 connection(s)
I20260810 11:47:37.532931 4860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:37.538985 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.536773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:40467)
0810 11:47:37.537744 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.538010 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":348,"threads_started":1}
W20260810 11:47:37.539791 5002 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:42899 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:42899: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
I20260810 11:47:37.540382 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 4860
I20260810 11:47:37.540894 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.540473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:38013)
0810 11:47:37.540708 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.540780 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 11:47:37.541114 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:41023
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:42899
--webserver_port=34975
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:47:37.541904 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:37.541541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:60319)
0810 11:47:37.541702 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:37.541792 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 11:47:37.542219 5002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:47:37.908221 5007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:37.908531 5007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:37.908767 5007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:37.916395 5007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:37.916519 5007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:37.916716 5007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:37.927217 5007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:41023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=34975
--tserver_master_addrs=127.20.93.190:42899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:37.928637 5007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:37.930167 5007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:37.945098 5013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:37.945052 5018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:37.945147 5016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:37.951424 5007 server_base.cc:1034] running on GCE node
I20260810 11:47:37.952310 5007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:37.953503 5007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:37.954818 5007 hybrid_clock.cc:648] HybridClock initialized: now 1786362457954770 us; error 51 us; skew 500 ppm
I20260810 11:47:37.955189 5007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:37 dist-test-slave-njxd krb5kdc[3385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362457, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:37.958741 5007 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:37.960543 5007 webserver.cc:460] Webserver started at http://127.20.93.131:34975/ using document root <none> and password file <none>
I20260810 11:47:37.961395 5007 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:37.961555 5007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:37.967520 5007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:47:37.970584 5025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:37.972267 5007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:37.972479 5007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "427852e89adc4b0482187764f5309331"
format_stamp: "Formatted at 2026-08-10 11:47:32 on dist-test-slave-njxd"
server_key: "2fa024e99843c1020bf51799097d3c32"
server_key_iv: "ce759ee2e76f512b887849ffc61b0867"
server_key_version: "encryptionkey@0"
I20260810 11:47:37.973554 5007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:37.987353 5007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:37.992261 5007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:37.992537 5007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:37.993659 5007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:37.995271 5007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:37.995368 5007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:37.995452 5007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:37.995508 5007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:38.018321 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.017288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:49489)
0810 11:47:38.017865 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.017962 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":301,"threads_started":1}
I20260810 11:47:38.033319 5007 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:41023
I20260810 11:47:38.033429 5148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:41023 every 8 connection(s)
I20260810 11:47:38.035051 5007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:38.035540 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5007
I20260810 11:47:38.040701 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.038605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:40189)
0810 11:47:38.039486 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.039767 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
W20260810 11:47:38.041466 5149 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:42899 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:42899: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
I20260810 11:47:38.042480 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.042029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:36717)
0810 11:47:38.042233 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.042352 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 11:47:38.043354 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.043017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:35679)
0810 11:47:38.043140 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.043214 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 11:47:38.043609 5149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 11:47:38.543901 5155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.542846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:53213)
0810 11:47:38.543388 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.543497 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":279,"threads_started":1}
I20260810 11:47:38.567395 5156 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:38.567726 5156 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:38.569427 5156 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
Aug 10 11:47:38 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
Aug 10 11:47:38 dist-test-slave-njxd krb5kdc[3385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:47:38.613854 5159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.570165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46779 (local address 127.20.93.131:41023)
0810 11:47:38.584998 (+ 14833us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:38.585025 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:38.586348 (+ 1323us) server_negotiation.cc:407] Connection header received
0810 11:47:38.588107 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:38.588117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:38.588613 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:38.588959 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:38.591268 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.592556 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.594589 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.595331 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.595552 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:38.606430 (+ 10878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:38.606438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:38.606452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:38.606496 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:38.609639 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:38.610557 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:38.610564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:38.610571 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:38.610742 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:38.611237 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:38.611243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:38.611247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:38.611914 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:38.612137 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:38.612972 (+ 835us) server_negotiation.cc:299] Negotiation successful
0810 11:47:38.613381 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14526,"thread_start_us":14362,"threads_started":1}
I20260810 11:47:38.613876 5158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.570015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:41023 (local address 127.20.93.129:46779)
0810 11:47:38.586213 (+ 16198us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.586229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:38.586434 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:38.589358 (+ 2924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:38.589364 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:38.590278 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:38.591033 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:38.591053 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.592747 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.592752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:38.594399 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:38.594413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.595610 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.595617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:38.595885 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:38.602740 (+ 6855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:38.602787 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:38.606061 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:38.609827 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:38.609838 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:38.609855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:38.610376 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:38.610870 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:38.610876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:38.610880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:38.611104 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:38.612071 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:38.612080 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:38.612388 (+ 308us) client_negotiation.cc:769] Sending connection context
0810 11:47:38.612736 (+ 348us) client_negotiation.cc:240] Negotiation successful
0810 11:47:38.613279 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16097,"mutex_wait_us":302,"spinlock_wait_cycles":63104,"thread_start_us":15704,"threads_started":1}
I20260810 11:47:38.613999 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.570009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:42965 (local address 127.20.93.129:53895)
0810 11:47:38.585601 (+ 15592us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:38.585742 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:38.586420 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:38.589077 (+ 2657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:38.589101 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:38.590278 (+ 1177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:38.591030 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:38.591053 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.592746 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.592750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:38.594397 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:38.594413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.595564 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.595570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:38.595891 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:38.602738 (+ 6847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:38.602769 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:38.605712 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:38.610083 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:38.610091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:38.610098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:38.610509 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:38.610978 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:38.610984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:38.610988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:38.611143 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:38.612705 (+ 1562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:38.612709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:38.612841 (+ 132us) client_negotiation.cc:769] Sending connection context
0810 11:47:38.612981 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 11:47:38.613460 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15405,"mutex_wait_us":248,"spinlock_wait_cycles":15872,"thread_start_us":15250,"threads_started":1}
I20260810 11:47:38.614125 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:38.570240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:53895 (local address 127.20.93.130:42965)
0810 11:47:38.585513 (+ 15273us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:38.585534 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:38.585927 (+ 393us) server_negotiation.cc:407] Connection header received
0810 11:47:38.588238 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:38.588245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:38.588599 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:38.588949 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:38.591260 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.592508 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.594597 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:38.595297 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:38.595531 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:38.606320 (+ 10789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:38.606337 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:38.606364 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:38.606420 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:38.609922 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:38.610674 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:38.610680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:38.610686 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:38.610850 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:38.611293 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:38.611302 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:38.611304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:38.611986 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:38.612646 (+ 660us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:38.613248 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 11:47:38.613723 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15069,"thread_start_us":14873,"threads_started":1}
W20260810 11:47:38.618686 4726 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
W20260810 11:47:38.619045 4727 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:38.619199 4727 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72bfd6559c554c69b88a50644afc95e7; no voters: 427852e89adc4b0482187764f5309331, 6497a6c9c19e4f4894e178ff338eec6c
I20260810 11:47:38.619663 5156 raft_consensus.cc:2726] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:47:39.019930 5158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:39.019263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:59975)
0810 11:47:39.019477 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:39.019601 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 11:47:39.045248 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:39.044199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:49321)
0810 11:47:39.044825 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:39.044993 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":317,"threads_started":1}
I20260810 11:47:39.545801 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:39.544981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:37461)
0810 11:47:39.545461 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:39.545553 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":261,"threads_started":1}
I20260810 11:47:40.022315 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:40.021033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:42333)
0810 11:47:40.021849 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:40.021992 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":406,"threads_started":1}
I20260810 11:47:40.047305 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:40.046498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:35009)
0810 11:47:40.047040 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:40.047138 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":273,"threads_started":1}
I20260810 11:47:40.547667 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:40.546891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:45613)
0810 11:47:40.547377 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:40.547469 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":261,"threads_started":1}
I20260810 11:47:40.865343 5166 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:40.865586 5166 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:40.866353 5166 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
W20260810 11:47:40.867414 4727 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
W20260810 11:47:40.867590 4726 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:40.867807 4726 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72bfd6559c554c69b88a50644afc95e7; no voters: 427852e89adc4b0482187764f5309331, 6497a6c9c19e4f4894e178ff338eec6c
I20260810 11:47:40.868116 5166 raft_consensus.cc:2726] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:47:41.024406 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:41.023351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:46929)
0810 11:47:41.024074 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:41.024208 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":391,"threads_started":1}
I20260810 11:47:41.049089 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:41.048358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:50791)
0810 11:47:41.048823 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:41.048911 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":239,"threads_started":1}
I20260810 11:47:41.550058 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:41.548961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:33833)
0810 11:47:41.549669 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:41.549812 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":335,"threads_started":1}
I20260810 11:47:42.026575 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:42.025717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:33875)
0810 11:47:42.026280 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:42.026372 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":240,"threads_started":1}
I20260810 11:47:42.051081 5171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:42.050225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:50495)
0810 11:47:42.050804 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:42.050893 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":317,"threads_started":1}
I20260810 11:47:42.552213 5172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:42.551192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:33177)
0810 11:47:42.551928 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:42.552025 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":453,"threads_started":1}
I20260810 11:47:43.028334 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:43.027561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:42559)
0810 11:47:43.028054 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:43.028143 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
I20260810 11:47:43.053314 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:43.052162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:36285)
0810 11:47:43.052910 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:43.053060 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":415,"threads_started":1}
I20260810 11:47:43.110742 5176 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:43.111019 5176 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:43.111799 5176 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
W20260810 11:47:43.113071 4726 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
W20260810 11:47:43.113286 4727 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:43.113417 4727 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72bfd6559c554c69b88a50644afc95e7; no voters: 427852e89adc4b0482187764f5309331, 6497a6c9c19e4f4894e178ff338eec6c
I20260810 11:47:43.113754 5176 raft_consensus.cc:2726] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:47:43.554162 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:43.553261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:38909)
0810 11:47:43.553880 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:43.553969 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":402,"threads_started":1}
I20260810 11:47:44.030364 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:44.029413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:38555)
0810 11:47:44.030026 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:44.030127 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":355,"threads_started":1}
I20260810 11:47:44.055231 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:44.054388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:39733)
0810 11:47:44.054958 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:44.055047 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":294,"threads_started":1}
I20260810 11:47:44.556017 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:44.555159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:44201)
0810 11:47:44.555684 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:44.555813 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":254,"threads_started":1}
I20260810 11:47:45.032356 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:45.031422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:42343)
0810 11:47:45.032010 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:45.032144 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":304,"threads_started":1}
I20260810 11:47:45.057267 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:45.056234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:60215)
0810 11:47:45.056956 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:45.057058 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":399,"threads_started":1}
I20260810 11:47:45.557931 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:45.557122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:33181)
0810 11:47:45.557661 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:45.557746 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":276,"threads_started":1}
I20260810 11:47:46.034345 5185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:46.033416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:56685)
0810 11:47:46.034068 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:46.034169 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":387,"threads_started":1}
I20260810 11:47:46.059427 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:46.058570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:37221)
0810 11:47:46.059150 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:46.059234 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":339,"threads_started":1}
I20260810 11:47:46.560184 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:46.559121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:37063)
0810 11:47:46.559845 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:46.559971 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":326,"threads_started":1}
I20260810 11:47:46.647534 5188 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:46.647758 5188 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:46.648532 5188 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
W20260810 11:47:46.649526 4727 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
W20260810 11:47:46.649792 4726 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:46.649938 4726 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72bfd6559c554c69b88a50644afc95e7; no voters: 427852e89adc4b0482187764f5309331, 6497a6c9c19e4f4894e178ff338eec6c
I20260810 11:47:46.650220 5188 raft_consensus.cc:2726] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:47:47.036700 5189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:47.035703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:54917)
0810 11:47:47.036346 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:47.036489 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":333,"threads_started":1}
I20260810 11:47:47.061868 5190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:47.060822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:59421)
0810 11:47:47.061486 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:47.061701 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":361,"threads_started":1}
I20260810 11:47:47.562407 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:47.561438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:57419)
0810 11:47:47.562074 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:47.562220 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":417,"threads_started":1}
I20260810 11:47:48.038723 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:48.037857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:60057)
0810 11:47:48.038415 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:48.038519 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":298,"threads_started":1}
I20260810 11:47:48.064059 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:48.063028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:56303)
0810 11:47:48.063763 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:48.063863 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":466,"threads_started":1}
I20260810 11:47:48.564431 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:48.563632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:43847)
0810 11:47:48.564155 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:48.564253 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":244,"threads_started":1}
I20260810 11:47:49.040575 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:49.039824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:50681)
0810 11:47:49.040298 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:49.040396 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260810 11:47:49.066073 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:49.065245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:37745)
0810 11:47:49.065806 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:49.065903 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":248,"threads_started":1}
I20260810 11:47:49.566442 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:49.565615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:33359)
0810 11:47:49.566151 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:49.566253 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":276,"threads_started":1}
I20260810 11:47:50.038100 5200 raft_consensus.cc:482] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:50.038295 5200 raft_consensus.cc:504] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "427852e89adc4b0482187764f5309331" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 41023 } } peers { permanent_uuid: "6497a6c9c19e4f4894e178ff338eec6c" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 42965 } } peers { permanent_uuid: "72bfd6559c554c69b88a50644afc95e7" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46065 } }
I20260810 11:47:50.038944 5200 leader_election.cc:288] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 427852e89adc4b0482187764f5309331 (127.20.93.131:41023), 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965)
W20260810 11:47:50.039965 4727 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 427852e89adc4b0482187764f5309331 (127.20.93.131:41023): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
W20260810 11:47:50.040380 4726 leader_election.cc:341] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6497a6c9c19e4f4894e178ff338eec6c (127.20.93.130:42965): Not found: Tablet not found: 108738e6f9674ca58f28c02d1de2769f
I20260810 11:47:50.040515 4726 leader_election.cc:302] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72bfd6559c554c69b88a50644afc95e7; no voters: 427852e89adc4b0482187764f5309331, 6497a6c9c19e4f4894e178ff338eec6c
I20260810 11:47:50.040830 5200 raft_consensus.cc:2726] T 108738e6f9674ca58f28c02d1de2769f P 72bfd6559c554c69b88a50644afc95e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:47:50.042164 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:50.041383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:55593)
0810 11:47:50.041867 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:50.041958 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":261,"threads_started":1}
I20260810 11:47:50.067813 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:50.067084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:51671)
0810 11:47:50.067541 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:50.067626 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260810 11:47:50.568617 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:50.567617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:35441)
0810 11:47:50.568300 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:50.568442 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":341,"threads_started":1}
I20260810 11:47:51.044039 5205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:51.043246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:59945)
0810 11:47:51.043743 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:51.043841 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260810 11:47:51.070150 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:51.069041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:36103)
0810 11:47:51.069718 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:51.069890 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":290,"threads_started":1}
I20260810 11:47:51.570907 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:51.569910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:55683)
0810 11:47:51.570494 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:51.570658 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":272,"threads_started":1}
I20260810 11:47:52.046285 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:52.045462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:45747)
0810 11:47:52.046008 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:52.046104 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":308,"threads_started":1}
I20260810 11:47:52.072260 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:52.071408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:49393)
0810 11:47:52.071928 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:52.072039 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":283,"threads_started":1}
I20260810 11:47:52.573202 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:52.572216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.130:55229)
0810 11:47:52.572891 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:52.573023 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":342,"threads_started":1}
I20260810 11:47:53.044955 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4669
I20260810 11:47:53.048450 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:53.047227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.129:53685)
0810 11:47:53.048083 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:53.048214 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":587,"threads_started":1}
I20260810 11:47:53.066337 5212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:53.046127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.20.93.129:46065)
0810 11:47:53.046782 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:53.046792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:53.046822 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:47:53.046892 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:53.046902 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:53.047173 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:53.047362 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:53.049969 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.050953 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.052261 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.052876 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.053030 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:53.059449 (+ 6419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:53.059466 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:53.059482 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:53.059528 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:53.063058 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.063945 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.063953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.063956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.064106 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.064575 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.064584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.064586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.064966 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:53.065310 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:53.065693 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:47:53.065994 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":284,"threads_started":1}
I20260810 11:47:53.067740 4850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:53.081924 5215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:53.080767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42899 (local address 127.20.93.131:43187)
0810 11:47:53.081406 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:53.081567 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":310,"threads_started":1}
I20260810 11:47:53.174710 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4669
I20260810 11:47:53.198148 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 4860
I20260810 11:47:53.214232 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:53.198890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.20.93.130:42965)
0810 11:47:53.199402 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:53.199410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:53.199431 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:47:53.199502 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:53.199506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:53.199566 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:53.199698 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:53.201165 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.202028 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.203365 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.203842 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.203981 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:53.207428 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:53.207434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:53.207437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:53.207479 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:53.210784 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.211727 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.211733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.211737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.211899 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.212335 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.212341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.212344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.212676 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:53.213009 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:53.213343 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 11:47:53.213893 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":258,"threads_started":1}
I20260810 11:47:53.214965 4996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:53.301105 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 4860
I20260810 11:47:53.323370 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5007
I20260810 11:47:53.339833 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:53.324374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.20.93.131:41023)
0810 11:47:53.325036 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:53.325041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:53.325059 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:47:53.325117 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:53.325121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:53.325192 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:53.325337 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:53.326985 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.327802 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.329167 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:53.329773 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:53.329923 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:53.333071 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:53.333077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:53.333080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:53.333123 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:53.336460 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.337443 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.337449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.337453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.337712 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:53.338165 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:53.338170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:53.338174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:53.338487 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:53.338782 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:53.339147 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 11:47:53.339396 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":366,"threads_started":1}
I20260810 11:47:53.340654 5143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:53.428709 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5007
2026-08-10T11:47:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25181 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:47:53 dist-test-slave-njxd krb5kdc[5222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:47:53 dist-test-slave-njxd krb5kdc[5222](info): set up 2 sockets
Aug 10 11:47:53 dist-test-slave-njxd krb5kdc[5222](info): commencing operation
krb5kdc: starting...
W20260810 11:47:54.631515 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:47:54 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:47:54.958107 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:55.329281 5237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:55.329707 5237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:55.329870 5237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:55.337867 5237 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:47:55.338004 5237 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:47:55.338092 5237 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:47:55.338188 5237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:47:55.338258 5237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:55.338326 5237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:55.338375 5237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:47:55.338430 5237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:47:55.349385 5237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:55.350837 5237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:55.352255 5237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:47:55.366896 5237 server_base.cc:1034] running on GCE node
W20260810 11:47:55.366896 5242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:55.366979 5243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:55.366928 5247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:55.368364 5237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:55.370154 5237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:55.371490 5237 hybrid_clock.cc:648] HybridClock initialized: now 1786362475371405 us; error 55 us; skew 500 ppm
I20260810 11:47:55.371918 5237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:55 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362475, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:55.375697 5237 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:47:55.377470 5237 webserver.cc:460] Webserver started at http://127.20.93.190:42737/ using document root <none> and password file <none>
I20260810 11:47:55.378338 5237 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:55.378484 5237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:55.378844 5237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:55.382239 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12"
format_stamp: "Formatted at 2026-08-10 11:47:55 on dist-test-slave-njxd"
server_key: "548af2c490758b226e9cd0776fbc781f"
server_key_iv: "b571ca99d8efbd331ab8be9c0b6ba838"
server_key_version: "encryptionkey@0"
I20260810 11:47:55.383184 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12"
format_stamp: "Formatted at 2026-08-10 11:47:55 on dist-test-slave-njxd"
server_key: "548af2c490758b226e9cd0776fbc781f"
server_key_iv: "b571ca99d8efbd331ab8be9c0b6ba838"
server_key_version: "encryptionkey@0"
I20260810 11:47:55.389056 5237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 11:47:55.393206 5254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:55.394819 5237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:55.395028 5237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12"
format_stamp: "Formatted at 2026-08-10 11:47:55 on dist-test-slave-njxd"
server_key: "548af2c490758b226e9cd0776fbc781f"
server_key_iv: "b571ca99d8efbd331ab8be9c0b6ba838"
server_key_version: "encryptionkey@0"
I20260810 11:47:55.395287 5237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:55.424774 5237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:55.429646 5237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:55.429951 5237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:55.450214 5237 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44307
I20260810 11:47:55.450217 5312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44307 every 8 connection(s)
I20260810 11:47:55.451813 5237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:47:55.456254 5313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:55.461175 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5237
I20260810 11:47:55.461414 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:47:55.462193 20854 external_mini_cluster.cc:1428] Setting key 7ea0d8eeba5fa10844b6fa5d45965235
I20260810 11:47:55.468766 5313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap starting.
I20260810 11:47:55.472746 5313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:55.474156 5313 log.cc:824] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:55.477349 5313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: No bootstrap required, opened a new log
Aug 10 11:47:55 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:55.489055 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:55.464426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.20.93.190:44307)
0810 11:47:55.465302 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:55.465314 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:55.465364 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:47:55.467283 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:55.467309 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:55.467747 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:55.468101 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:55.470208 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:55.471648 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:55.473680 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:55.474495 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:55.474652 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:55.479381 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:55.479392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:55.479421 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:55.479475 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:55.483876 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:55.484657 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:55.484667 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:55.484679 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:55.484922 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:55.485304 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:55.485310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:55.485314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:55.486220 (+ 906us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:55.486575 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:55.487038 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 11:47:55.487573 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":403,"threads_started":1}
I20260810 11:47:55.488747 5313 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:47:55.489266 5313 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:55.489388 5313 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Initialized, Role: FOLLOWER
I20260810 11:47:55.490175 5313 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:47:55.490495 5313 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:47:55.490661 5313 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:47:55.490890 5313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:55.493625 5313 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:47:55.494179 5313 leader_election.cc:302] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc5bc3b4b20a4e509f2ffa928eb8ec12; no voters:
I20260810 11:47:55.494843 5313 leader_election.cc:288] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:47:55.495085 5318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:55.495811 5318 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 LEADER]: Becoming Leader. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Running, Role: LEADER
I20260810 11:47:55.496392 5318 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:47:55.497044 5313 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:47:55.499773 5319 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:47:55.500464 5319 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:55.499768 5320 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc5bc3b4b20a4e509f2ffa928eb8ec12. Latest consensus state: current_term: 1 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:47:55.501614 5320 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:47:55.501724 5326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:47:55.509454 5326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:47:55.520723 5326 catalog_manager.cc:1347] Generated new cluster ID: a26a9e0647c94d9c85b47b6b5096e611
I20260810 11:47:55.520862 5326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:47:55.550488 5326 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:47:55.551934 5326 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:47:55.559142 5326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Generated new TSK 0
I20260810 11:47:55.559993 5326 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:47:55.642488 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:56.016865 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:56.017166 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:56.017342 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:56.025072 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:56.025216 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:56.025398 5342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:47:56.035822 5342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:56.037251 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:56.038748 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:56.053855 5349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:56.053820 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:56.053920 5351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:56.054728 5342 server_base.cc:1034] running on GCE node
I20260810 11:47:56.055510 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:56.057085 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:56.058375 5342 hybrid_clock.cc:648] HybridClock initialized: now 1786362476058336 us; error 73 us; skew 500 ppm
I20260810 11:47:56.058780 5342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:56 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362476, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:56.062592 5342 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:47:56.064329 5342 webserver.cc:460] Webserver started at http://127.20.93.129:39135/ using document root <none> and password file <none>
I20260810 11:47:56.065128 5342 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:56.065266 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:56.065706 5342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:56.069025 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "18927438e2414942b95a81694f423065"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "f61d5190d4b6ad91b1b25e45bb8d0650"
server_key_iv: "659c8ca7d5096d4d3348c7f2ee08e1cf"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.070170 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "18927438e2414942b95a81694f423065"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "f61d5190d4b6ad91b1b25e45bb8d0650"
server_key_iv: "659c8ca7d5096d4d3348c7f2ee08e1cf"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.076517 5342 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260810 11:47:56.080672 5359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.082291 5342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:47:56.082486 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "18927438e2414942b95a81694f423065"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "f61d5190d4b6ad91b1b25e45bb8d0650"
server_key_iv: "659c8ca7d5096d4d3348c7f2ee08e1cf"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.082708 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:56.103734 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:56.108537 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:56.108881 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:56.110033 5342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:56.111642 5342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:56.111724 5342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.111892 5342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:56.111958 5342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.149104 5342 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:45597
I20260810 11:47:56.149158 5484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:45597 every 8 connection(s)
I20260810 11:47:56.150606 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:47:56.156668 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5342
I20260810 11:47:56.156893 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:47:56.157619 20854 external_mini_cluster.cc:1428] Setting key dc377bbafe9c87bb9b98746f91a72c7a
Aug 10 11:47:56 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362476, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:56.174376 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:56.153374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:51871 (local address 127.20.93.190:44307)
0810 11:47:56.153883 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:56.153889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:56.155014 (+ 1125us) server_negotiation.cc:407] Connection header received
0810 11:47:56.156970 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:56.156977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:56.157078 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:56.157206 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:56.159243 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.160316 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.162084 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.162832 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.162979 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:56.167347 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:56.167352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:56.167356 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:56.167392 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:56.170291 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:56.171394 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:56.171399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:56.171401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:56.171531 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:56.172130 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:56.172134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:56.172136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:56.172426 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:56.172555 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:56.173562 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0810 11:47:56.173837 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
I20260810 11:47:56.175463 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:56.154061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:51871)
0810 11:47:56.154840 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:56.154880 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:56.156628 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:56.157482 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:56.157491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:56.158156 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:56.159036 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:56.159063 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.160517 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.160523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:56.161931 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:56.161945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.163076 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.163081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:56.163309 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:56.164092 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:56.164123 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:56.167059 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:56.170471 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:56.170485 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:56.170508 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:56.171250 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:56.171638 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:56.171645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:56.171651 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:56.171966 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:56.172573 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:56.172585 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:56.173099 (+ 514us) client_negotiation.cc:769] Sending connection context
0810 11:47:56.173462 (+ 363us) client_negotiation.cc:240] Negotiation successful
0810 11:47:56.174139 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":290,"threads_started":1}
I20260810 11:47:56.177245 5485 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:47:56.177670 5485 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:56.178686 5485 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:47:56.181257 5275 ts_manager.cc:194] Registered new tserver with Master: 18927438e2414942b95a81694f423065 (127.20.93.129:45597)
I20260810 11:47:56.182904 5275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:51871
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:47:56.231137 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:56.593281 5492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:56.593670 5492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:56.593789 5492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:56.601704 5492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:56.601858 5492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:56.602005 5492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:47:56.612120 5492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:56.613454 5492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:56.614914 5492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:56.628652 5497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:56.628749 5500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:56.628698 5498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:56.629763 5492 server_base.cc:1034] running on GCE node
I20260810 11:47:56.630455 5492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:56.631541 5492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:56.632858 5492 hybrid_clock.cc:648] HybridClock initialized: now 1786362476632791 us; error 52 us; skew 500 ppm
I20260810 11:47:56.633246 5492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:56 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362476, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:56.636873 5492 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:47:56.638670 5492 webserver.cc:460] Webserver started at http://127.20.93.130:37487/ using document root <none> and password file <none>
I20260810 11:47:56.639497 5492 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:56.639688 5492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:56.640045 5492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:56.643443 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "754df297a61b4df8b88d037b82da7d3b"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "b8d6ee1b73b0f16b6a833abd4a9e4e01"
server_key_iv: "83803aa76c983bac297322bcfd7a2487"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.644328 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "754df297a61b4df8b88d037b82da7d3b"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "b8d6ee1b73b0f16b6a833abd4a9e4e01"
server_key_iv: "83803aa76c983bac297322bcfd7a2487"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.650717 5492 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:47:56.654489 5507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.655958 5492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:47:56.656152 5492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "754df297a61b4df8b88d037b82da7d3b"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "b8d6ee1b73b0f16b6a833abd4a9e4e01"
server_key_iv: "83803aa76c983bac297322bcfd7a2487"
server_key_version: "encryptionkey@0"
I20260810 11:47:56.656366 5492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:56.679270 5492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:56.684335 5492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:56.684675 5492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:56.685961 5492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:56.687463 5492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:56.687577 5492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.687728 5492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:56.687798 5492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:56.724288 5492 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:43499
I20260810 11:47:56.724383 5644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:43499 every 8 connection(s)
I20260810 11:47:56.725687 5492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:47:56.734711 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5492
I20260810 11:47:56.734922 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:47:56.735427 20854 external_mini_cluster.cc:1428] Setting key 92fcc431599adb4140a9109760b4642b
Aug 10 11:47:56 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362476, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:56.748131 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:56.728588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:34443 (local address 127.20.93.190:44307)
0810 11:47:56.729017 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:56.729022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:56.730103 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 11:47:56.732031 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:56.732037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:56.732131 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:56.732250 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:56.733903 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.734939 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.736806 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.737365 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.737507 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:56.741025 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:56.741030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:56.741034 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:56.741067 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:56.743741 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:56.744550 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:56.744559 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:56.744563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:56.744762 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:56.745435 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:56.745442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:56.745448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:56.745895 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:56.746063 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:56.747213 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0810 11:47:56.747622 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":211,"threads_started":1}
I20260810 11:47:56.748930 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:56.729187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:34443)
0810 11:47:56.729943 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:56.729981 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:56.731750 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:56.732499 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:56.732508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:56.733056 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:56.733723 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:56.733749 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.735112 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.735119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:56.736640 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:56.736653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:56.737626 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:56.737635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:56.737864 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:56.738554 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:56.738577 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:56.740771 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:56.743901 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:56.743911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:56.743932 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:56.744416 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:56.744948 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:56.744972 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:56.744978 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:56.745278 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:56.746098 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:56.746111 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:56.746641 (+ 530us) client_negotiation.cc:769] Sending connection context
0810 11:47:56.747028 (+ 387us) client_negotiation.cc:240] Negotiation successful
0810 11:47:56.747616 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":346,"threads_started":1}
I20260810 11:47:56.750550 5645 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:47:56.750931 5645 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:56.751709 5645 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:47:56.753448 5275 ts_manager.cc:194] Registered new tserver with Master: 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
I20260810 11:47:56.754362 5275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:34443
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:47:56.805686 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:47:57.170996 5651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:47:57.171345 5651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:47:57.171511 5651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:47:57.179240 5651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:47:57.179397 5651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:47:57.179574 5651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:47:57.186473 5485 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
I20260810 11:47:57.190270 5651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:47:57.191644 5651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:47:57.193064 5651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:47:57.207240 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:57.207512 5658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:47:57.208143 5660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:47:57.208467 5651 server_base.cc:1034] running on GCE node
I20260810 11:47:57.209159 5651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:47:57.210284 5651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:47:57.211535 5651 hybrid_clock.cc:648] HybridClock initialized: now 1786362477211484 us; error 43 us; skew 500 ppm
I20260810 11:47:57.211913 5651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:47:57 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362477, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:47:57.215560 5651 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:47:57.217388 5651 webserver.cc:460] Webserver started at http://127.20.93.131:37275/ using document root <none> and password file <none>
I20260810 11:47:57.218374 5651 fs_manager.cc:359] Metadata directory not provided
I20260810 11:47:57.218529 5651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:47:57.218891 5651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:47:57.222090 5651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
format_stamp: "Formatted at 2026-08-10 11:47:57 on dist-test-slave-njxd"
server_key: "d19f902b770ed5c05be7f0e1dc78b52a"
server_key_iv: "b5ebbfed75ce36e14916e0bb0e8860f1"
server_key_version: "encryptionkey@0"
I20260810 11:47:57.222885 5651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
format_stamp: "Formatted at 2026-08-10 11:47:57 on dist-test-slave-njxd"
server_key: "d19f902b770ed5c05be7f0e1dc78b52a"
server_key_iv: "b5ebbfed75ce36e14916e0bb0e8860f1"
server_key_version: "encryptionkey@0"
I20260810 11:47:57.229475 5651 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:47:57.233891 5667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:57.235533 5651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:47:57.235699 5651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
format_stamp: "Formatted at 2026-08-10 11:47:57 on dist-test-slave-njxd"
server_key: "d19f902b770ed5c05be7f0e1dc78b52a"
server_key_iv: "b5ebbfed75ce36e14916e0bb0e8860f1"
server_key_version: "encryptionkey@0"
I20260810 11:47:57.235921 5651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:47:57.267081 5651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:47:57.271935 5651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:47:57.272197 5651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:47:57.273190 5651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:47:57.274710 5651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:47:57.274797 5651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:57.274892 5651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:47:57.274938 5651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:47:57.310791 5651 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:34967
I20260810 11:47:57.310895 5798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:34967 every 8 connection(s)
I20260810 11:47:57.312206 5651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:47:57.318676 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5651
I20260810 11:47:57.318892 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:47:57.319387 20854 external_mini_cluster.cc:1428] Setting key fbb5ba015d24ffea71cddacbf6529f00
Aug 10 11:47:57 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362477, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:47:57.334431 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.315032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:47285 (local address 127.20.93.190:44307)
0810 11:47:57.315491 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.315497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.316409 (+ 912us) server_negotiation.cc:407] Connection header received
0810 11:47:57.318425 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.318430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.318530 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.318653 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:57.320648 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.321849 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.323776 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.324178 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.324279 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.327989 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:57.327995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:57.327998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:57.328033 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:57.330970 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:57.331788 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:57.331792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:57.331795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:57.331926 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:57.332328 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:57.332332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:57.332334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:57.332642 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:57.332762 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.333810 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.334060 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":247,"threads_started":1}
I20260810 11:47:57.335521 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.315498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:47285)
0810 11:47:57.316227 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.316278 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.318116 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.318918 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.318927 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.319596 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:47:57.320414 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.320449 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.322017 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.322022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.323641 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.323656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.324378 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.324383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.324582 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.325351 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:47:57.325374 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:47:57.327779 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:47:57.331152 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:57.331160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:57.331175 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:57.331654 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:57.332023 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:47:57.332026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:47:57.332029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:47:57.332231 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:47:57.332743 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:47:57.332749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:47:57.333150 (+ 401us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.333486 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.334077 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":279,"threads_started":1}
I20260810 11:47:57.337040 5799 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:47:57.337344 5799 heartbeater.cc:464] Registering TS with master...
I20260810 11:47:57.338199 5799 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:47:57.339746 5275 ts_manager.cc:194] Registered new tserver with Master: b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967)
I20260810 11:47:57.340682 5275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:47285
I20260810 11:47:57.348420 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:47:57.367208 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.353378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.20.93.190:44307)
0810 11:47:57.353662 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.353667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.354053 (+ 386us) server_negotiation.cc:407] Connection header received
0810 11:47:57.354186 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.354190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.354263 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.354361 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:57.355996 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.356784 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.358353 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.358777 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.358887 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.361759 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:57.361765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:57.361769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:57.361808 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:57.364516 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:57.365205 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:57.365210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:57.365212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:57.365340 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:57.365715 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:57.365718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:57.365720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:57.366059 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:57.366254 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.366782 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.366973 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:47:57.371898 5275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:47:57.379323 5275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:47:57.420284 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.393127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:34967 (local address 127.0.0.1:49566)
0810 11:47:57.395604 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.395657 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.395902 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.412401 (+ 16499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.412409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.412444 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:57.412823 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.412834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.414956 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.414962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.417198 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.417213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.419180 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.419186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.419395 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.419455 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.419580 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.419810 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2106,"mutex_wait_us":1,"thread_start_us":1940,"threads_started":1}
I20260810 11:47:57.421031 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.396287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:38856)
0810 11:47:57.413543 (+ 17256us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.413563 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.413834 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.414474 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.414481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.414507 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:57.414915 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.414928 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.416498 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.416505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.418475 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.418488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.420101 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.420111 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.420295 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.420330 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.420418 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.420551 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17114,"mutex_wait_us":1,"thread_start_us":13626,"threads_started":1}
I20260810 11:47:57.421547 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.393212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:43499 (local address 127.0.0.1:34624)
0810 11:47:57.409557 (+ 16345us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.409606 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.409795 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.413500 (+ 3705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.413505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.413536 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:57.413971 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.413986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.415847 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.415853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.418250 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.418266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.420698 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.420704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.420831 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.420854 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.420912 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.421000 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16157,"mutex_wait_us":1,"thread_start_us":19962,"threads_started":1}
I20260810 11:47:57.421851 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.397013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.20.93.129:45597)
0810 11:47:57.403498 (+ 6485us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.403512 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.413712 (+ 10200us) server_negotiation.cc:407] Connection header received
0810 11:47:57.414045 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.414057 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.414266 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.414459 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:57.415114 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.416358 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.418790 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.419876 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.420039 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.420309 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.421298 (+ 989us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.421471 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6230,"thread_start_us":5723,"threads_started":1}
I20260810 11:47:57.421851 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.393885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.20.93.131:34967)
0810 11:47:57.410862 (+ 16977us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.410877 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.410927 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:47:57.411078 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.411091 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.411635 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.411918 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:57.413043 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.414810 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.417382 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.418968 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.420440 (+ 1472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.421044 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.421131 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.421476 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16729,"thread_start_us":16513,"threads_started":1}
I20260810 11:47:57.422096 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.393208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.20.93.130:43499)
0810 11:47:57.412785 (+ 19577us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.412797 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.412833 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:47:57.412950 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.412962 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.413261 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.413528 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:57.414156 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.415698 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.418454 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.420020 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.420160 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.420395 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.421307 (+ 912us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.421493 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19342,"thread_start_us":19169,"threads_started":1}
I20260810 11:47:57.426040 5725 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac (DEFAULT_TABLE table=test-table [id=61837d70a3d5454e9ddc112bb706d083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:57.426051 5411 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac (DEFAULT_TABLE table=test-table [id=61837d70a3d5454e9ddc112bb706d083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:57.426051 5560 tablet_service.cc:1467] Processing CreateTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac (DEFAULT_TABLE table=test-table [id=61837d70a3d5454e9ddc112bb706d083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:47:57.427543 5560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2a5bbf12c453b9f5ec7996ed897ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:57.427541 5411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2a5bbf12c453b9f5ec7996ed897ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:57.427577 5725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f2a5bbf12c453b9f5ec7996ed897ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:47:57.440485 5820 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Bootstrap starting.
I20260810 11:47:57.442880 5822 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Bootstrap starting.
I20260810 11:47:57.444654 5821 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Bootstrap starting.
I20260810 11:47:57.446173 5820 tablet_bootstrap.cc:654] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:57.446894 5822 tablet_bootstrap.cc:654] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:57.447620 5820 log.cc:824] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:57.447701 5821 tablet_bootstrap.cc:654] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Neither blocks nor log segments found. Creating new log.
I20260810 11:47:57.448166 5822 log.cc:824] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:57.448876 5821 log.cc:824] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Log is configured to *not* fsync() on all Append() calls
I20260810 11:47:57.451411 5820 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: No bootstrap required, opened a new log
I20260810 11:47:57.451690 5820 ts_tablet_manager.cc:1397] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260810 11:47:57.451845 5822 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: No bootstrap required, opened a new log
I20260810 11:47:57.452142 5822 ts_tablet_manager.cc:1397] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 11:47:57.452648 5821 tablet_bootstrap.cc:492] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: No bootstrap required, opened a new log
I20260810 11:47:57.452976 5821 ts_tablet_manager.cc:1397] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 11:47:57.462855 5820 raft_consensus.cc:348] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.463395 5820 raft_consensus.cc:374] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:57.463500 5820 raft_consensus.cc:729] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016c6e7a08d4d64ba1f589f19cbf7e0, State: Initialized, Role: FOLLOWER
I20260810 11:47:57.463774 5822 raft_consensus.cc:348] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.464244 5822 raft_consensus.cc:374] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:57.464373 5822 raft_consensus.cc:729] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754df297a61b4df8b88d037b82da7d3b, State: Initialized, Role: FOLLOWER
I20260810 11:47:57.464306 5820 consensus_queue.cc:260] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.465040 5822 consensus_queue.cc:260] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.465018 5821 raft_consensus.cc:348] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.465673 5821 raft_consensus.cc:374] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:47:57.465857 5821 raft_consensus.cc:729] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18927438e2414942b95a81694f423065, State: Initialized, Role: FOLLOWER
I20260810 11:47:57.466753 5821 consensus_queue.cc:260] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.473058 5820 ts_tablet_manager.cc:1428] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Time spent starting tablet: real 0.021s user 0.021s sys 0.000s
I20260810 11:47:57.473881 5799 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
I20260810 11:47:57.475297 5822 ts_tablet_manager.cc:1428] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Time spent starting tablet: real 0.023s user 0.018s sys 0.005s
I20260810 11:47:57.475951 5645 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
W20260810 11:47:57.477977 5646 tablet.cc:2369] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:57.481700 5821 ts_tablet_manager.cc:1428] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Time spent starting tablet: real 0.029s user 0.015s sys 0.013s
W20260810 11:47:57.564055 5800 tablet.cc:2369] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:47:57.653364 5486 tablet.cc:2369] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:47:57.860777 5826 raft_consensus.cc:482] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:47:57.861166 5826 raft_consensus.cc:504] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.863045 5826 leader_election.cc:288] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499), 18927438e2414942b95a81694f423065 (127.20.93.129:45597)
I20260810 11:47:57.869962 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.863678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:57003 (local address 127.20.93.129:45597)
0810 11:47:57.863937 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.863946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.864530 (+ 584us) server_negotiation.cc:407] Connection header received
0810 11:47:57.864623 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.864627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.864715 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.864820 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:57.865428 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.866190 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.868151 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.868949 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.869162 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.869245 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.869519 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.869681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:47:57.870116 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.863584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.20.93.131:57003)
0810 11:47:57.864395 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.864410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.864526 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.864845 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.864848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.864868 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:57.865244 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.865251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.866335 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.866345 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.867965 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.867979 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.869157 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.869163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.869352 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.869380 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.869493 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.869672 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":23,"thread_start_us":529,"threads_started":1}
I20260810 11:47:57.870180 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.863749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:49565 (local address 127.20.93.130:43499)
0810 11:47:57.863968 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.863975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.863997 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:47:57.864254 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.864265 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.864332 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.864386 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:47:57.865281 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.866084 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.868338 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.869176 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.869358 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.869435 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.869786 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.869916 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:47:57.870348 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.863269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:43499 (local address 127.20.93.131:49565)
0810 11:47:57.863853 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 11:47:57.863875 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:47:57.864054 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:47:57.864618 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:47:57.864622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:47:57.864652 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:47:57.865089 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.865104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.866219 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.866227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.868177 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:47:57.868189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.869399 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.869405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:47:57.869568 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.869646 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 11:47:57.869742 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 11:47:57.869889 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":41,"thread_start_us":735,"threads_started":1}
I20260810 11:47:57.871628 5437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2a5bbf12c453b9f5ec7996ed897ac" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18927438e2414942b95a81694f423065" is_pre_election: true
I20260810 11:47:57.872220 5437 raft_consensus.cc:2454] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in term 0.
I20260810 11:47:57.872118 5592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2a5bbf12c453b9f5ec7996ed897ac" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b" is_pre_election: true
I20260810 11:47:57.872668 5592 raft_consensus.cc:2454] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in term 0.
I20260810 11:47:57.873037 5671 leader_election.cc:302] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18927438e2414942b95a81694f423065, b016c6e7a08d4d64ba1f589f19cbf7e0; no voters:
I20260810 11:47:57.873623 5826 raft_consensus.cc:2781] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:47:57.873808 5826 raft_consensus.cc:482] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:47:57.873919 5826 raft_consensus.cc:3037] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:57.876247 5826 raft_consensus.cc:504] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.876935 5826 leader_election.cc:288] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 election: Requested vote from peers 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499), 18927438e2414942b95a81694f423065 (127.20.93.129:45597)
I20260810 11:47:57.877460 5592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2a5bbf12c453b9f5ec7996ed897ac" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b"
I20260810 11:47:57.877507 5437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f2a5bbf12c453b9f5ec7996ed897ac" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18927438e2414942b95a81694f423065"
I20260810 11:47:57.877705 5592 raft_consensus.cc:3037] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:57.877800 5437 raft_consensus.cc:3037] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:47:57.880036 5592 raft_consensus.cc:2454] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in term 1.
I20260810 11:47:57.880045 5437 raft_consensus.cc:2454] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in term 1.
I20260810 11:47:57.880520 5673 leader_election.cc:302] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 754df297a61b4df8b88d037b82da7d3b, b016c6e7a08d4d64ba1f589f19cbf7e0; no voters:
I20260810 11:47:57.880824 5826 raft_consensus.cc:2781] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:47:57.881337 5826 raft_consensus.cc:686] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 LEADER]: Becoming Leader. State: Replica: b016c6e7a08d4d64ba1f589f19cbf7e0, State: Running, Role: LEADER
I20260810 11:47:57.881902 5826 consensus_queue.cc:237] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } }
I20260810 11:47:57.886616 5275 catalog_manager.cc:5495] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } health_report { overall_health: UNKNOWN } } }
I20260810 11:47:57.975718 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:57.969089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.20.93.131:34967)
0810 11:47:57.969692 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:57.969697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:57.969783 (+ 86us) server_negotiation.cc:407] Connection header received
0810 11:47:57.970045 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:57.970051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:57.970133 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:57.970293 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:47:57.970767 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.971509 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.973201 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:57.973742 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:57.973853 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:57.974425 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:47:57.975013 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:47:57.975141 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:57.975236 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 11:47:57.975315 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
I20260810 11:47:57.984210 5437 raft_consensus.cc:1261] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Refusing update from remote peer b016c6e7a08d4d64ba1f589f19cbf7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:57.984210 5592 raft_consensus.cc:1261] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Refusing update from remote peer b016c6e7a08d4d64ba1f589f19cbf7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:47:57.985163 5833 consensus_queue.cc:1035] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:57.985524 5826 consensus_queue.cc:1035] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:47:57.998962 5842 mvcc.cc:205] Tried to move back new op lower bound from 7316940709813149696 to 7316940709408137216. Current Snapshot: MvccSnapshot[applied={T|T < 7316940709813149696}]
I20260810 11:47:58.012254 5274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:58.012583 5274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:47:58.016538 5274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Sending DeleteTablet for 3 replicas of tablet 07f2a5bbf12c453b9f5ec7996ed897ac
I20260810 11:47:58.017868 5560 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:58 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:34624
I20260810 11:47:58.017932 5411 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:58 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:38856
I20260810 11:47:58.018069 5725 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f2a5bbf12c453b9f5ec7996ed897ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:47:58 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:49566
I20260810 11:47:58.018687 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5342
I20260810 11:47:58.030426 5852 tablet_replica.cc:330] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: stopping tablet replica
I20260810 11:47:58.031252 5851 tablet_replica.cc:330] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: stopping tablet replica
I20260810 11:47:58.032163 5851 raft_consensus.cc:2229] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:58.032907 5852 raft_consensus.cc:2229] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:47:58.034066 5852 raft_consensus.cc:2258] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:58.034363 5851 raft_consensus.cc:2258] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:47:58.037442 5852 ts_tablet_manager.cc:1905] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:58.039795 5851 ts_tablet_manager.cc:1905] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:58.040551 5853 tablet_replica.cc:330] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: stopping tablet replica
I20260810 11:47:58.041105 5853 raft_consensus.cc:2229] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:47:58.042208 5853 raft_consensus.cc:2258] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 11:47:58 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:47:58.047313 5853 ts_tablet_manager.cc:1905] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:47:58.047959 5852 ts_tablet_manager.cc:1918] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:58.048228 5852 log.cc:1196] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/07f2a5bbf12c453b9f5ec7996ed897ac
I20260810 11:47:58.048842 5852 ts_tablet_manager.cc:1939] T 07f2a5bbf12c453b9f5ec7996ed897ac P 754df297a61b4df8b88d037b82da7d3b: Deleting consensus metadata
I20260810 11:47:58.051254 5258 catalog_manager.cc:4841] TS 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499): tablet 07f2a5bbf12c453b9f5ec7996ed897ac (table test-table [id=61837d70a3d5454e9ddc112bb706d083]) successfully deleted
I20260810 11:47:58.054939 5851 ts_tablet_manager.cc:1918] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:58.055197 5851 log.cc:1196] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/07f2a5bbf12c453b9f5ec7996ed897ac
I20260810 11:47:58.055851 5851 ts_tablet_manager.cc:1939] T 07f2a5bbf12c453b9f5ec7996ed897ac P 18927438e2414942b95a81694f423065: Deleting consensus metadata
I20260810 11:47:58.056753 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:58.020186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.20.93.129:45597)
0810 11:47:58.020425 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:58.020434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:58.032858 (+ 12424us) server_negotiation.cc:407] Connection header received
0810 11:47:58.033405 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:58.033414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:58.033521 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:58.033629 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:58.035881 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.037084 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.039303 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.040033 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.040148 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:58.046549 (+ 6401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:58.046569 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:58.046589 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:58.046641 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:58.051959 (+ 5318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.053172 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.053187 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.053191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.053385 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.053993 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.053999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.054002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.054539 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:58.055291 (+ 752us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:58.056020 (+ 729us) server_negotiation.cc:299] Negotiation successful
0810 11:47:58.056327 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:47:58.057158 5853 ts_tablet_manager.cc:1918] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:47:58.057426 5853 log.cc:1196] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/07f2a5bbf12c453b9f5ec7996ed897ac
I20260810 11:47:58.058053 5257 catalog_manager.cc:4841] TS 18927438e2414942b95a81694f423065 (127.20.93.129:45597): tablet 07f2a5bbf12c453b9f5ec7996ed897ac (table test-table [id=61837d70a3d5454e9ddc112bb706d083]) successfully deleted
I20260810 11:47:58.058508 5480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:58.058560 5853 ts_tablet_manager.cc:1939] T 07f2a5bbf12c453b9f5ec7996ed897ac P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting consensus metadata
I20260810 11:47:58.060782 5257 catalog_manager.cc:4841] TS b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967): tablet 07f2a5bbf12c453b9f5ec7996ed897ac (table test-table [id=61837d70a3d5454e9ddc112bb706d083]) successfully deleted
I20260810 11:47:58.150862 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5342
I20260810 11:47:58.171787 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5492
Aug 10 11:47:58 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:47:58.194540 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:58.172742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.20.93.130:43499)
0810 11:47:58.172933 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:58.172937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:58.173007 (+ 70us) server_negotiation.cc:407] Connection header received
0810 11:47:58.173237 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:58.173241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:58.173311 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:58.173421 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:58.175405 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.176176 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.178424 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.178929 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.179047 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:58.188045 (+ 8998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:58.188063 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:58.188080 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:58.188140 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:58.191260 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.192145 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.192154 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.192156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.192299 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.192761 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.192765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.192767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.193192 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:58.193539 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:58.193925 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 11:47:58.194176 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:47:58.195356 5640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:58.321095 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5492
I20260810 11:47:58.347353 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5651
Aug 10 11:47:58 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:47:58.368000 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:47:58.348396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.20.93.131:34967)
0810 11:47:58.348619 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:47:58.348626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:47:58.348649 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:47:58.348718 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:47:58.348729 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:47:58.348821 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:47:58.348951 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:47:58.351132 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.352413 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.354415 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:47:58.355183 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:47:58.355324 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:47:58.360097 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:47:58.360116 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:47:58.360137 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:47:58.360191 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:47:58.364738 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.365562 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.365606 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.365610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.365794 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:47:58.366201 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:47:58.366208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:47:58.366212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:47:58.366740 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:47:58.367063 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 11:47:58.367275 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 11:47:58.367546 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:47:58.368861 5794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:58.512074 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5651
I20260810 11:47:58.538093 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5237
I20260810 11:47:58.539773 5308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:47:58.660894 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5237
I20260810 11:48:18.691990 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--webserver_interface=127.20.93.190
--webserver_port=42737
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:48:19.073837 5859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:19.074355 5859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:19.074550 5859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:19.082945 5859 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:48:19.083073 5859 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:48:19.083117 5859 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:48:19.083225 5859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:48:19.083297 5859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:19.083421 5859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:19.083509 5859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:48:19.083596 5859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:48:19.094724 5859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=42737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:19.096232 5859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:19.097759 5859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:19.126624 5870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:19.126660 5867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:19.126935 5866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:19.127954 5859 server_base.cc:1034] running on GCE node
I20260810 11:48:19.128875 5859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:19.130668 5859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:19.131964 5859 hybrid_clock.cc:648] HybridClock initialized: now 1786362499131883 us; error 56 us; skew 500 ppm
I20260810 11:48:19.132431 5859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:19 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:19.140408 5859 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:48:19.142560 5859 webserver.cc:460] Webserver started at http://127.20.93.190:42737/ using document root <none> and password file <none>
I20260810 11:48:19.143464 5859 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:19.143633 5859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:19.150360 5859 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:48:19.153903 5877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:19.155694 5859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:48:19.155941 5859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12"
format_stamp: "Formatted at 2026-08-10 11:47:55 on dist-test-slave-njxd"
server_key: "548af2c490758b226e9cd0776fbc781f"
server_key_iv: "b571ca99d8efbd331ab8be9c0b6ba838"
server_key_version: "encryptionkey@0"
I20260810 11:48:19.157145 5859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:19.177757 5859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:19.183008 5859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:19.183370 5859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:19.204923 5859 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44307
I20260810 11:48:19.204943 5938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44307 every 8 connection(s)
I20260810 11:48:19.206768 5859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:48:19.209333 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5859
I20260810 11:48:19.210600 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:45597
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=39135
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:48:19.218247 5939 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:48:19.224292 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap starting.
I20260810 11:48:19.246589 5939 log.cc:824] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Log is configured to *not* fsync() on all Append() calls
I20260810 11:48:19.262251 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:48:19.262916 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap complete.
I20260810 11:48:19.273840 5939 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:19.275532 5939 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Initialized, Role: FOLLOWER
I20260810 11:48:19.276575 5939 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:19.276988 5939 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:48:19.277165 5939 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:48:19.277390 5939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:48:19.284296 5939 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:19.284996 5939 leader_election.cc:302] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc5bc3b4b20a4e509f2ffa928eb8ec12; no voters:
I20260810 11:48:19.285820 5939 leader_election.cc:288] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:48:19.285974 5944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:48:19.288743 5939 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:48:19.289911 5944 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 LEADER]: Becoming Leader. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Running, Role: LEADER
I20260810 11:48:19.290671 5944 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:19.293877 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc5bc3b4b20a4e509f2ffa928eb8ec12. Latest consensus state: current_term: 2 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:48:19.294312 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:48:19.294945 5945 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:48:19.295512 5945 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:48:19.305936 5957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:48:19.313279 5957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:48:19.314827 5957 catalog_manager.cc:1259] Loaded cluster ID: a26a9e0647c94d9c85b47b6b5096e611
I20260810 11:48:19.314975 5957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:48:19.318215 5957 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:48:19.319651 5957 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Loaded TSK: 0
I20260810 11:48:19.342695 5957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Generated new TSK 1
I20260810 11:48:19.343147 5957 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:48:19.624044 5942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:19.624405 5942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:19.624629 5942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:19.632309 5942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:19.632496 5942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:19.632696 5942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:48:19.643178 5942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:45597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=39135
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:19.644675 5942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:19.646215 5942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:19.662114 5971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:19.662220 5973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:19.662660 5942 server_base.cc:1034] running on GCE node
W20260810 11:48:19.662799 5970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:19.663664 5942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:19.665895 5942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:19.667168 5942 hybrid_clock.cc:648] HybridClock initialized: now 1786362499667098 us; error 55 us; skew 500 ppm
I20260810 11:48:19.667706 5942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:19 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:19.671350 5942 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:48:19.673228 5942 webserver.cc:460] Webserver started at http://127.20.93.129:39135/ using document root <none> and password file <none>
I20260810 11:48:19.674135 5942 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:19.674305 5942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:19.680323 5942 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:48:19.683602 5981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:19.685235 5942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:48:19.685417 5942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "18927438e2414942b95a81694f423065"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "f61d5190d4b6ad91b1b25e45bb8d0650"
server_key_iv: "659c8ca7d5096d4d3348c7f2ee08e1cf"
server_key_version: "encryptionkey@0"
I20260810 11:48:19.686753 5942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:19.723807 5942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:19.728634 5942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:19.728942 5942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:19.730064 5942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:19.731530 5942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:48:19.731634 5942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:19.731743 5942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:19.731845 5942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:19.769862 5942 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:45597
I20260810 11:48:19.769974 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:45597 every 8 connection(s)
I20260810 11:48:19.771585 5942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:48:19.772330 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 5942
I20260810 11:48:19.773319 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:43499
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=37487
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:48:19 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:48:19.800814 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:19.775146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:35661 (local address 127.20.93.190:44307)
0810 11:48:19.775858 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:19.775878 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:19.776670 (+ 792us) server_negotiation.cc:407] Connection header received
0810 11:48:19.779138 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:19.779162 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:19.779720 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:19.780211 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:19.782108 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:19.784211 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:19.786070 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:19.787239 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:19.787459 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:19.791946 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:19.791961 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:19.791987 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:19.792050 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:19.795768 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:19.796643 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:19.796650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:19.796656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:19.796834 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:19.797332 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:19.797336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:19.797338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:19.797926 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:19.798142 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:19.798984 (+ 842us) server_negotiation.cc:299] Negotiation successful
0810 11:48:19.799418 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":276,"threads_started":1}
I20260810 11:48:19.800814 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:19.775439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:35661)
0810 11:48:19.776444 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:19.776495 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:19.778462 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:19.780427 (+ 1965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:19.780436 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:19.781079 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:48:19.781828 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:19.781851 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:19.784439 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:19.784443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:19.785864 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:19.785878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:19.787494 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:19.787499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:19.787716 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:19.788470 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:48:19.788499 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:48:19.791553 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:48:19.795949 (+ 4396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:19.795961 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:19.795985 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:19.796459 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:19.796963 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:19.796968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:19.796971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:19.797188 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:19.798086 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:48:19.798095 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:48:19.798447 (+ 352us) client_negotiation.cc:769] Sending connection context
0810 11:48:19.798735 (+ 288us) client_negotiation.cc:240] Negotiation successful
0810 11:48:19.799358 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":431,"threads_started":1}
I20260810 11:48:19.804605 6106 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:48:19.805027 6106 heartbeater.cc:464] Registering TS with master...
I20260810 11:48:19.806185 6106 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:48:19.809209 5900 ts_manager.cc:194] Registered new tserver with Master: 18927438e2414942b95a81694f423065 (127.20.93.129:45597)
I20260810 11:48:19.811425 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:35661
W20260810 11:48:20.149909 6110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:20.150244 6110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:20.150349 6110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:20.158355 6110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:20.158491 6110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:20.158618 6110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:48:20.169173 6110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:43499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=37487
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:20.170701 6110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:20.172160 6110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:20.187417 6117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:20.187587 6120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:20.187587 6118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:20.188807 6110 server_base.cc:1034] running on GCE node
I20260810 11:48:20.189567 6110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:20.190876 6110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:20.192112 6110 hybrid_clock.cc:648] HybridClock initialized: now 1786362500192038 us; error 69 us; skew 500 ppm
I20260810 11:48:20.192555 6110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:20 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:20.196679 6110 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:48:20.198550 6110 webserver.cc:460] Webserver started at http://127.20.93.130:37487/ using document root <none> and password file <none>
I20260810 11:48:20.199390 6110 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:20.199580 6110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:20.206410 6110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:48:20.209872 6128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.212000 6110 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:48:20.212280 6110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "754df297a61b4df8b88d037b82da7d3b"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "b8d6ee1b73b0f16b6a833abd4a9e4e01"
server_key_iv: "83803aa76c983bac297322bcfd7a2487"
server_key_version: "encryptionkey@0"
I20260810 11:48:20.213483 6110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:20.230398 6110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:20.235584 6110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:20.235924 6110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:20.237121 6110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:20.238808 6110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:48:20.238955 6110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.239099 6110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:20.239195 6110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.277405 6110 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:43499
I20260810 11:48:20.277468 6253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:43499 every 8 connection(s)
I20260810 11:48:20.279273 6110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:48:20.280922 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6110
I20260810 11:48:20.281824 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:34967
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=37275
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:48:20 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:48:20.309338 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.283057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:51423 (local address 127.20.93.190:44307)
0810 11:48:20.283300 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.283309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.284863 (+ 1554us) server_negotiation.cc:407] Connection header received
0810 11:48:20.288212 (+ 3349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.288220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.288384 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.288507 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:20.291278 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.292406 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.295159 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.295760 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.295939 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.301360 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:20.301367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:20.301370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:20.301415 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:20.304834 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.306153 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.306157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.306160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.306314 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.307007 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.307012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.307014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.307495 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:20.307665 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.308814 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.309067 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:48:20.311195 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.283653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:51423)
0810 11:48:20.284625 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:20.284689 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:20.287748 (+ 3059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:20.288869 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:20.288885 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:20.289930 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:48:20.290983 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.291020 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.292666 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.292680 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.294912 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.294936 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.296067 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.296075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.296353 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.297239 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:48:20.297290 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:48:20.300820 (+ 3530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:48:20.305080 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:20.305094 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:20.305119 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:20.305973 (+ 854us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:20.306443 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:20.306451 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:20.306457 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:20.306814 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:20.307673 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:48:20.307686 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:48:20.308264 (+ 578us) client_negotiation.cc:769] Sending connection context
0810 11:48:20.308646 (+ 382us) client_negotiation.cc:240] Negotiation successful
0810 11:48:20.309351 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":356,"threads_started":1}
I20260810 11:48:20.313246 6255 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:48:20.313733 6255 heartbeater.cc:464] Registering TS with master...
I20260810 11:48:20.314944 6255 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:48:20.317381 5900 ts_manager.cc:194] Registered new tserver with Master: 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
I20260810 11:48:20.318841 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:51423
W20260810 11:48:20.671965 6260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:20.672303 6260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:20.672405 6260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:20.680122 6260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:20.680280 6260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:20.680464 6260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:48:20.691284 6260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:34967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=37275
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:20.692677 6260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:20.694381 6260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:20.708678 6270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:20.708631 6267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:20.708724 6266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:20.709637 6260 server_base.cc:1034] running on GCE node
I20260810 11:48:20.710368 6260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:20.711705 6260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:20.712985 6260 hybrid_clock.cc:648] HybridClock initialized: now 1786362500712926 us; error 39 us; skew 500 ppm
I20260810 11:48:20.713389 6260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:20 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:20.716864 6260 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:48:20.718670 6260 webserver.cc:460] Webserver started at http://127.20.93.131:37275/ using document root <none> and password file <none>
I20260810 11:48:20.719677 6260 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:20.719816 6260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:20.726399 6260 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:48:20.729930 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.731647 6260 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:48:20.731894 6260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
format_stamp: "Formatted at 2026-08-10 11:47:57 on dist-test-slave-njxd"
server_key: "d19f902b770ed5c05be7f0e1dc78b52a"
server_key_iv: "b5ebbfed75ce36e14916e0bb0e8860f1"
server_key_version: "encryptionkey@0"
I20260810 11:48:20.733002 6260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:20.757220 6260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:20.762249 6260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:20.762563 6260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:20.763693 6260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:20.765201 6260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:48:20.765332 6260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.765472 6260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:20.765561 6260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:20.803370 6260 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:34967
I20260810 11:48:20.803467 6402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:34967 every 8 connection(s)
I20260810 11:48:20.804972 6260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:48:20.813911 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6260
I20260810 11:48:20.815876 6106 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
Aug 10 11:48:20 dist-test-slave-njxd krb5kdc[5222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:48:20.830241 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.807896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:49911 (local address 127.20.93.190:44307)
0810 11:48:20.808110 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.808115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.809399 (+ 1284us) server_negotiation.cc:407] Connection header received
0810 11:48:20.811482 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.811487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.811567 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.811668 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:20.813493 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.814705 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.817379 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.818240 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.818414 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.822904 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:20.822912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:20.822918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:20.822965 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:20.826408 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.827373 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.827379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.827383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.827552 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.828039 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.828045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.828048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.828437 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:20.828587 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.829532 (+ 945us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.829886 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:48:20.831547 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.808361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:49911)
0810 11:48:20.809208 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:20.809249 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:20.811171 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:20.811941 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:20.811950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:20.812536 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:48:20.813244 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.813266 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.814933 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.814939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.817158 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.817177 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.818601 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.818609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.819048 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.819856 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:48:20.819890 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:48:20.822597 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:48:20.826582 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:20.826589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:20.826610 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:20.827223 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:20.827686 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:48:20.827689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:48:20.827692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:48:20.827903 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:48:20.828593 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:48:20.828602 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:48:20.829047 (+ 445us) client_negotiation.cc:769] Sending connection context
0810 11:48:20.829430 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 11:48:20.830051 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":369,"threads_started":1}
I20260810 11:48:20.832638 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.815054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.20.93.190:44307)
0810 11:48:20.816217 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.816223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.816245 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:48:20.816300 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.816305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.816381 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.816493 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:20.818774 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.819884 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.821409 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.822147 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.822284 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.825244 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:20.825251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:20.825257 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:20.825319 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:20.828814 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.829737 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.829742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.829745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.829889 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.830495 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.830500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.830503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.830817 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:20.831023 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.831434 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.831681 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260810 11:48:20.833374 6403 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44307
I20260810 11:48:20.833806 6403 heartbeater.cc:464] Registering TS with master...
I20260810 11:48:20.834640 6403 heartbeater.cc:511] Master 127.20.93.190:44307 requested a full tablet report, sending...
I20260810 11:48:20.836181 5900 ts_manager.cc:194] Registered new tserver with Master: b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967)
I20260810 11:48:20.837047 5900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:49911
I20260810 11:48:20.848896 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:48:20.857007 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.850116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.20.93.190:44307)
0810 11:48:20.850325 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.850330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.850975 (+ 645us) server_negotiation.cc:407] Connection header received
0810 11:48:20.851235 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.851241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.851423 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.851572 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:48:20.852085 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.852838 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.855004 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.855466 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.855572 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.856111 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:48:20.856417 (+ 306us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 11:48:20.856775 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34424: token expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 11:48:20.857124 6407 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34424: token expired
W20260810 11:48:20.857884 5807 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.93.190:44307)
I20260810 11:48:20.872841 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.858483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.20.93.190:44307)
0810 11:48:20.858666 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.858674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.858693 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:48:20.858919 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.858924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.858997 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.859102 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:20.860744 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.861681 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.863174 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.863584 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.863725 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.866827 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:20.866832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:20.866836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:20.866873 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:20.869805 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.870625 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.870631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.870635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.870789 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:20.871224 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:20.871229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:20.871234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:20.871586 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:20.871860 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.872231 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.872483 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:48:20.876852 5900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:48:20.884956 5900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:48:20.920136 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.899082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:34967 (local address 127.0.0.1:59014)
0810 11:48:20.900208 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:20.900270 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:20.900468 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:20.910590 (+ 10122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:20.910600 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:20.910633 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:20.911203 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.911219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.914010 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.914016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.916494 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.916514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.919003 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.919009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.919236 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.919297 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 11:48:20.919451 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 11:48:20.919672 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":1,"thread_start_us":366,"threads_started":1}
I20260810 11:48:20.920147 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.899410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.20.93.131:34967)
0810 11:48:20.900440 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.900452 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.900491 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:48:20.909407 (+ 8916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.909422 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.909877 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.910154 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:20.911918 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.913805 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.916703 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.918391 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.918538 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.918809 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.919539 (+ 730us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.919748 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":857,"thread_start_us":1382,"threads_started":1}
I20260810 11:48:20.921247 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.901053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.20.93.130:43499)
0810 11:48:20.913165 (+ 12112us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.913176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.913210 (+ 34us) server_negotiation.cc:407] Connection header received
0810 11:48:20.913317 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.913325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.913643 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.913900 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:20.914595 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.915877 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.918275 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.919749 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.919899 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.920102 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.920741 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.920916 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11884,"thread_start_us":12137,"threads_started":1}
I20260810 11:48:20.921319 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.900395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:43499 (local address 127.0.0.1:54934)
0810 11:48:20.911368 (+ 10973us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:20.911389 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:20.911505 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:20.913885 (+ 2380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:20.913890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:20.913920 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:20.914390 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.914401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.916041 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.916047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.918047 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.918066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.920380 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.920388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.920520 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.920545 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:48:20.920677 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 11:48:20.920830 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10805,"mutex_wait_us":1,"thread_start_us":10255,"threads_started":1}
I20260810 11:48:20.921833 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.900379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:49308)
0810 11:48:20.910398 (+ 10019us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:20.910423 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:20.910555 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:20.912496 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:20.912504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:20.912539 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:20.912950 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.912963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.915385 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.915395 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.918128 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:20.918143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.920660 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.920667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:20.920841 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.920892 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 11:48:20.920992 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 11:48:20.921153 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9848,"mutex_wait_us":159,"thread_start_us":12090,"threads_started":1}
I20260810 11:48:20.921991 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:20.901021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.20.93.129:45597)
0810 11:48:20.911082 (+ 10061us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:20.911097 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:20.911142 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:48:20.911280 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:20.911293 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:20.911666 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:20.911964 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:20.913456 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.915211 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.918363 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:20.920044 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:20.920207 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:20.920455 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:20.921422 (+ 967us) server_negotiation.cc:299] Negotiation successful
0810 11:48:20.921652 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9770,"thread_start_us":10347,"threads_started":1}
I20260810 11:48:20.925309 6033 tablet_service.cc:1467] Processing CreateTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 (DEFAULT_TABLE table=test-table [id=6cb05204ddea4074b861e7941e6ad25c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:48:20.925362 6331 tablet_service.cc:1467] Processing CreateTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 (DEFAULT_TABLE table=test-table [id=6cb05204ddea4074b861e7941e6ad25c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:48:20.925773 6182 tablet_service.cc:1467] Processing CreateTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 (DEFAULT_TABLE table=test-table [id=6cb05204ddea4074b861e7941e6ad25c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:48:20.927001 6331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42092d1a2b374f0d9568f8ed596b5de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:48:20.927016 6182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42092d1a2b374f0d9568f8ed596b5de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:48:20.927042 6033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42092d1a2b374f0d9568f8ed596b5de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:48:20.940960 6421 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Bootstrap starting.
I20260810 11:48:20.945261 6421 tablet_bootstrap.cc:654] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Neither blocks nor log segments found. Creating new log.
I20260810 11:48:20.945536 6423 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Bootstrap starting.
I20260810 11:48:20.947434 6421 log.cc:824] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Log is configured to *not* fsync() on all Append() calls
I20260810 11:48:20.948654 6422 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Bootstrap starting.
I20260810 11:48:20.949765 6423 tablet_bootstrap.cc:654] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Neither blocks nor log segments found. Creating new log.
I20260810 11:48:20.950978 6421 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: No bootstrap required, opened a new log
I20260810 11:48:20.951228 6423 log.cc:824] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:48:20.951331 6421 ts_tablet_manager.cc:1397] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 11:48:20.952477 6422 tablet_bootstrap.cc:654] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Neither blocks nor log segments found. Creating new log.
I20260810 11:48:20.953926 6422 log.cc:824] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:48:20.955127 6423 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: No bootstrap required, opened a new log
I20260810 11:48:20.955590 6423 ts_tablet_manager.cc:1397] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260810 11:48:20.960357 6421 raft_consensus.cc:348] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.960865 6421 raft_consensus.cc:374] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:48:20.960908 6422 tablet_bootstrap.cc:492] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: No bootstrap required, opened a new log
I20260810 11:48:20.960971 6421 raft_consensus.cc:729] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18927438e2414942b95a81694f423065, State: Initialized, Role: FOLLOWER
I20260810 11:48:20.961221 6422 ts_tablet_manager.cc:1397] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260810 11:48:20.961802 6421 consensus_queue.cc:260] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.963821 6421 ts_tablet_manager.cc:1428] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260810 11:48:20.966867 6423 raft_consensus.cc:348] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.967367 6423 raft_consensus.cc:374] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:48:20.967476 6423 raft_consensus.cc:729] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754df297a61b4df8b88d037b82da7d3b, State: Initialized, Role: FOLLOWER
I20260810 11:48:20.968232 6423 consensus_queue.cc:260] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.969565 6255 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
I20260810 11:48:20.970275 6423 ts_tablet_manager.cc:1428] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260810 11:48:20.973250 6422 raft_consensus.cc:348] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.973680 6422 raft_consensus.cc:374] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:48:20.973807 6422 raft_consensus.cc:729] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016c6e7a08d4d64ba1f589f19cbf7e0, State: Initialized, Role: FOLLOWER
I20260810 11:48:20.974556 6422 consensus_queue.cc:260] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:20.975718 6403 heartbeater.cc:503] Master 127.20.93.190:44307 was elected leader, sending a full tablet report...
I20260810 11:48:20.976408 6422 ts_tablet_manager.cc:1428] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260810 11:48:21.022877 6427 raft_consensus.cc:482] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:48:21.023276 6427 raft_consensus.cc:504] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
W20260810 11:48:21.024488 6107 tablet.cc:2369] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:48:21.024750 6429 raft_consensus.cc:482] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:48:21.025087 6429 raft_consensus.cc:504] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:21.025537 6427 leader_election.cc:288] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967), 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
I20260810 11:48:21.027705 6429 leader_election.cc:288] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18927438e2414942b95a81694f423065 (127.20.93.129:45597), 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
W20260810 11:48:21.031767 6256 tablet.cc:2369] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:48:21.036926 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.025887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:52659 (local address 127.20.93.131:34967)
0810 11:48:21.026280 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.026291 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.027270 (+ 979us) server_negotiation.cc:407] Connection header received
0810 11:48:21.028193 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.028200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.028267 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.028319 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:21.029261 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.030486 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.033219 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.035324 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.035511 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.035637 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.036488 (+ 851us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.036624 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":551,"server-negotiator.queue_time_us":231}
I20260810 11:48:21.037063 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.026233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:34967 (local address 127.20.93.129:52659)
0810 11:48:21.027166 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.027185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:21.028024 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:21.028522 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:21.028526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:21.028568 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:21.029010 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.029024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.030673 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.030680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.033025 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.033043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.035764 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.035770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.036132 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.036286 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 11:48:21.036463 (+ 177us) client_negotiation.cc:240] Negotiation successful
0810 11:48:21.036626 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":213,"thread_start_us":592,"threads_started":1}
I20260810 11:48:21.038331 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.026038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:49883 (local address 127.20.93.130:43499)
0810 11:48:21.026258 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.026265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.027025 (+ 760us) server_negotiation.cc:407] Connection header received
0810 11:48:21.027391 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.027401 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.027492 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.027572 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:21.029757 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.031289 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.034480 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.035786 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.035945 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.036032 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.037890 (+ 1858us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.038063 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:48:21.038376 6353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "18927438e2414942b95a81694f423065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" is_pre_election: true
I20260810 11:48:21.038961 6353 raft_consensus.cc:2454] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18927438e2414942b95a81694f423065 in term 0.
I20260810 11:48:21.039347 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.025901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:43499 (local address 127.20.93.129:49883)
0810 11:48:21.026883 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.026907 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:21.027162 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:21.029004 (+ 1842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:21.029008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:21.029037 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:21.029491 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.029508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.031528 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.031536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.034281 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.034299 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.036137 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.036148 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.037674 (+ 1526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.037721 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 11:48:21.038691 (+ 970us) client_negotiation.cc:240] Negotiation successful
0810 11:48:21.038889 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":2,"thread_start_us":567,"threads_started":1}
I20260810 11:48:21.039353 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.028065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:44263 (local address 127.20.93.130:43499)
0810 11:48:21.029181 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.029189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.029213 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:48:21.029272 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.029276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.029352 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.029732 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:21.030384 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.032999 (+ 2615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.035934 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.037548 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.038663 (+ 1115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.038745 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.038832 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.038967 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":720,"threads_started":1}
I20260810 11:48:21.039937 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.027640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:43499 (local address 127.20.93.131:44263)
0810 11:48:21.028027 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.028045 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:21.028156 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:21.029506 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:21.029511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:21.029547 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:21.030170 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.030184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.033220 (+ 3036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.033225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.035730 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.035747 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.037770 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.037776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.038024 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.038064 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:48:21.038151 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 11:48:21.038309 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":455}
I20260810 11:48:21.039847 5985 leader_election.cc:302] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18927438e2414942b95a81694f423065, b016c6e7a08d4d64ba1f589f19cbf7e0; no voters:
I20260810 11:48:21.040371 6427 raft_consensus.cc:2781] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:48:21.040553 6427 raft_consensus.cc:482] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:48:21.040686 6427 raft_consensus.cc:3037] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:48:21.040715 6204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "18927438e2414942b95a81694f423065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b" is_pre_election: true
I20260810 11:48:21.041352 6204 raft_consensus.cc:2454] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18927438e2414942b95a81694f423065 in term 0.
I20260810 11:48:21.041409 6202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b" is_pre_election: true
I20260810 11:48:21.041805 6202 raft_consensus.cc:2454] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in term 0.
I20260810 11:48:21.042325 6284 leader_election.cc:302] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 754df297a61b4df8b88d037b82da7d3b, b016c6e7a08d4d64ba1f589f19cbf7e0; no voters:
I20260810 11:48:21.042666 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.028419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:51337 (local address 127.20.93.129:45597)
0810 11:48:21.028634 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.028640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.034365 (+ 5725us) server_negotiation.cc:407] Connection header received
0810 11:48:21.034668 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.034674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.034784 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.034845 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:48:21.035932 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.037271 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.039976 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.041379 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.041593 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.041674 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.042255 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.042400 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":65}
I20260810 11:48:21.042819 6429 raft_consensus.cc:2781] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:48:21.042981 6429 raft_consensus.cc:482] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:48:21.043073 6429 raft_consensus.cc:3037] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:48:21.043414 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.027688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.20.93.131:51337)
0810 11:48:21.034248 (+ 6560us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.034269 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:48:21.034493 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:48:21.035044 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:48:21.035053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:48:21.035089 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:48:21.035709 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.035722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.037445 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.037451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.039771 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:48:21.039795 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.041822 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.041830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:48:21.042043 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.042098 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 11:48:21.042744 (+ 646us) client_negotiation.cc:240] Negotiation successful
0810 11:48:21.042901 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6380,"mutex_wait_us":126,"thread_start_us":6100,"threads_started":1}
I20260810 11:48:21.044598 6427 raft_consensus.cc:504] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:21.044579 6054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18927438e2414942b95a81694f423065" is_pre_election: true
I20260810 11:48:21.045323 6054 raft_consensus.cc:2379] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in current term 1: Already voted for candidate 18927438e2414942b95a81694f423065 in this term.
I20260810 11:48:21.045801 6429 raft_consensus.cc:504] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
I20260810 11:48:21.046619 6353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "18927438e2414942b95a81694f423065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
I20260810 11:48:21.046613 6202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "18927438e2414942b95a81694f423065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b"
I20260810 11:48:21.046777 6429 leader_election.cc:288] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 election: Requested vote from peers 18927438e2414942b95a81694f423065 (127.20.93.129:45597), 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
I20260810 11:48:21.046844 6202 raft_consensus.cc:3037] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:48:21.046953 6353 raft_consensus.cc:2379] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 18927438e2414942b95a81694f423065 in current term 1: Already voted for candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in this term.
I20260810 11:48:21.047039 6427 leader_election.cc:288] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [CANDIDATE]: Term 1 election: Requested vote from peers b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967), 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499)
I20260810 11:48:21.047236 6054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18927438e2414942b95a81694f423065"
I20260810 11:48:21.047492 6054 raft_consensus.cc:2379] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b016c6e7a08d4d64ba1f589f19cbf7e0 in current term 1: Already voted for candidate 18927438e2414942b95a81694f423065 in this term.
I20260810 11:48:21.047693 6204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42092d1a2b374f0d9568f8ed596b5de9" candidate_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754df297a61b4df8b88d037b82da7d3b"
I20260810 11:48:21.048478 6284 leader_election.cc:302] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b016c6e7a08d4d64ba1f589f19cbf7e0; no voters: 18927438e2414942b95a81694f423065, 754df297a61b4df8b88d037b82da7d3b
I20260810 11:48:21.048794 6429 raft_consensus.cc:2726] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 11:48:21.049826 6202 raft_consensus.cc:2454] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18927438e2414942b95a81694f423065 in term 1.
I20260810 11:48:21.050272 5986 leader_election.cc:302] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18927438e2414942b95a81694f423065, 754df297a61b4df8b88d037b82da7d3b; no voters: b016c6e7a08d4d64ba1f589f19cbf7e0
I20260810 11:48:21.050688 6427 raft_consensus.cc:2781] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:48:21.051230 6427 raft_consensus.cc:686] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 LEADER]: Becoming Leader. State: Replica: 18927438e2414942b95a81694f423065, State: Running, Role: LEADER
I20260810 11:48:21.051810 6427 consensus_queue.cc:237] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } }
W20260810 11:48:21.056763 6404 tablet.cc:2369] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:48:21.056771 5900 catalog_manager.cc:5495] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18927438e2414942b95a81694f423065 (127.20.93.129). New cstate: current_term: 1 leader_uuid: "18927438e2414942b95a81694f423065" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18927438e2414942b95a81694f423065" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 45597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 } health_report { overall_health: UNKNOWN } } }
I20260810 11:48:21.110395 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.104233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.20.93.129:45597)
0810 11:48:21.104445 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.104454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.104542 (+ 88us) server_negotiation.cc:407] Connection header received
0810 11:48:21.104861 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.104865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.104980 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.105157 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:48:21.105570 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.106430 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.107955 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.108390 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.108544 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.109096 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:48:21.109733 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:48:21.109897 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.109979 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.110070 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:48:21.119380 6202 raft_consensus.cc:1261] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Refusing update from remote peer 18927438e2414942b95a81694f423065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:48:21.119437 6353 raft_consensus.cc:1261] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Refusing update from remote peer 18927438e2414942b95a81694f423065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:48:21.120368 6434 consensus_queue.cc:1035] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:48:21.120822 6427 consensus_queue.cc:1035] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754df297a61b4df8b88d037b82da7d3b" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 43499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:48:21.148540 5900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:48:21.148847 5900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:48:21.152822 5900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Sending DeleteTablet for 3 replicas of tablet 42092d1a2b374f0d9568f8ed596b5de9
I20260810 11:48:21.154332 6033 tablet_service.cc:1514] Processing DeleteTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:48:21 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:49308
I20260810 11:48:21.154327 6331 tablet_service.cc:1514] Processing DeleteTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:48:21 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:59014
I20260810 11:48:21.154582 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5942
I20260810 11:48:21.155445 6182 tablet_service.cc:1514] Processing DeleteTablet for tablet 42092d1a2b374f0d9568f8ed596b5de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:48:21 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:54934
I20260810 11:48:21.155607 6450 tablet_replica.cc:330] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: stopping tablet replica
I20260810 11:48:21.156165 6450 raft_consensus.cc:2229] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:48:21.156730 6450 raft_consensus.cc:2258] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:48:21.157611 6451 tablet_replica.cc:330] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: stopping tablet replica
I20260810 11:48:21.158094 6451 raft_consensus.cc:2229] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:48:21.158811 6451 raft_consensus.cc:2258] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:48:21.160277 6450 ts_tablet_manager.cc:1905] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:48:21.165949 6451 ts_tablet_manager.cc:1905] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:48:21.170938 6450 ts_tablet_manager.cc:1918] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:48:21.171309 6450 log.cc:1196] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/42092d1a2b374f0d9568f8ed596b5de9
I20260810 11:48:21.172096 6450 ts_tablet_manager.cc:1939] T 42092d1a2b374f0d9568f8ed596b5de9 P b016c6e7a08d4d64ba1f589f19cbf7e0: Deleting consensus metadata
I20260810 11:48:21.174835 5880 catalog_manager.cc:4841] TS b016c6e7a08d4d64ba1f589f19cbf7e0 (127.20.93.131:34967): tablet 42092d1a2b374f0d9568f8ed596b5de9 (table test-table [id=6cb05204ddea4074b861e7941e6ad25c]) successfully deleted
I20260810 11:48:21.178735 6452 tablet_replica.cc:330] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: stopping tablet replica
I20260810 11:48:21.179293 6452 raft_consensus.cc:2229] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:48:21.179558 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.156260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.20.93.129:45597)
0810 11:48:21.156459 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.156465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.156484 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:48:21.156541 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.156548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.156640 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.156748 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:21.158935 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.160091 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.162191 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.162933 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.163080 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.170557 (+ 7477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:21.170573 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:21.170594 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:21.170657 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:21.175098 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.176148 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.176163 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.176180 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.176448 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.176989 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.176997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.177007 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.177816 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:21.178639 (+ 823us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.178822 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.179123 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:48:21.180012 6452 raft_consensus.cc:2258] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:48:21.180548 6099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:21.182508 6452 ts_tablet_manager.cc:1905] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:48:21.188580 6452 ts_tablet_manager.cc:1918] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:48:21.188764 6452 log.cc:1196] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/42092d1a2b374f0d9568f8ed596b5de9
I20260810 11:48:21.189327 6452 ts_tablet_manager.cc:1939] T 42092d1a2b374f0d9568f8ed596b5de9 P 754df297a61b4df8b88d037b82da7d3b: Deleting consensus metadata
I20260810 11:48:21.191258 5882 catalog_manager.cc:4841] TS 754df297a61b4df8b88d037b82da7d3b (127.20.93.130:43499): tablet 42092d1a2b374f0d9568f8ed596b5de9 (table test-table [id=6cb05204ddea4074b861e7941e6ad25c]) successfully deleted
I20260810 11:48:21.283010 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5942
W20260810 11:48:21.306299 5880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.93.129:45597 (error 108)
I20260810 11:48:21.307096 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 6110
I20260810 11:48:21.309191 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.308062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:49334)
0810 11:48:21.308845 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.309000 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":646}
W20260810 11:48:21.309518 5880 catalog_manager.cc:4584] TS 18927438e2414942b95a81694f423065 (127.20.93.129:45597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42092d1a2b374f0d9568f8ed596b5de9: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
I20260810 11:48:21.324754 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.308208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.20.93.130:43499)
0810 11:48:21.308428 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.308435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.308501 (+ 66us) server_negotiation.cc:407] Connection header received
0810 11:48:21.308726 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.308732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.308864 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.308975 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:21.310886 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.311803 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.313206 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.313875 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.314021 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.317181 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:21.317200 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:21.317220 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:21.317294 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:21.321141 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.322326 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.322336 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.322340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.322524 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.323042 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.323049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.323052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.323610 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:21.323975 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.324269 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.324478 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 11:48:21.325516 6247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:21.363363 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.362864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:49336)
0810 11:48:21.363093 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.363206 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 11:48:21.364472 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.364089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:49346)
0810 11:48:21.364240 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.364346 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 11:48:21.425094 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.424520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:56172)
0810 11:48:21.424815 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.424913 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260810 11:48:21.426295 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.425926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:56174)
0810 11:48:21.426085 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.426170 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 11:48:21.432504 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 6110
I20260810 11:48:21.455372 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 6260
I20260810 11:48:21.471117 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.456218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.20.93.131:34967)
0810 11:48:21.456458 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:21.456465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:21.456486 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:48:21.456708 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:21.456713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:21.456782 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:21.456894 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:21.458443 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.459229 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.460561 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:21.460985 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:21.461140 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:21.464382 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:21.464399 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:21.464417 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:21.464465 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:21.467627 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.468498 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.468506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.468508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.468669 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:21.469150 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:21.469161 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:21.469163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:21.469717 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:21.470049 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:21.470414 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 11:48:21.470720 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:48:21.471807 6397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:21.506850 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.506206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:56186)
0810 11:48:21.506534 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.506643 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 11:48:21.508255 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.507764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:56192)
0810 11:48:21.508011 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.508119 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260810 11:48:21.568441 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 6260
I20260810 11:48:21.590761 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 5859
I20260810 11:48:21.592083 5934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:21.716518 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:21.715089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:45597 (local address 127.0.0.1:56208)
0810 11:48:21.715950 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:21.716152 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:45597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":697}
I20260810 11:48:21.716802 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 5859
I20260810 11:48:41.744525 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--webserver_interface=127.20.93.190
--webserver_port=42737
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:48:42.142763 6458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:42.143086 6458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:42.143181 6458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:42.150861 6458 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:48:42.150979 6458 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:48:42.151018 6458 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:48:42.151073 6458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:48:42.151132 6458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:42.151197 6458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:42.151243 6458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:48:42.151292 6458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:48:42.161793 6458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=42737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:42.163187 6458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:42.164556 6458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:42.177798 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:42.177754 6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:42.177835 6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:42.178628 6458 server_base.cc:1034] running on GCE node
I20260810 11:48:42.179536 6458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:42.181067 6458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:42.182334 6458 hybrid_clock.cc:648] HybridClock initialized: now 1786362522182272 us; error 44 us; skew 500 ppm
I20260810 11:48:42.182724 6458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:42 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:42.186081 6458 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:48:42.187644 6458 webserver.cc:460] Webserver started at http://127.20.93.190:42737/ using document root <none> and password file <none>
I20260810 11:48:42.188364 6458 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:42.188453 6458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:42.194263 6458 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:48:42.197325 6474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:42.198987 6458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:48:42.199162 6458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12"
format_stamp: "Formatted at 2026-08-10 11:47:55 on dist-test-slave-njxd"
server_key: "548af2c490758b226e9cd0776fbc781f"
server_key_iv: "b571ca99d8efbd331ab8be9c0b6ba838"
server_key_version: "encryptionkey@0"
I20260810 11:48:42.200176 6458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:42.227245 6458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:42.231623 6458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:42.231889 6458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:42.252411 6458 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44307
I20260810 11:48:42.252456 6530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44307 every 8 connection(s)
I20260810 11:48:42.254220 6458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:48:42.262096 6531 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:48:42.264707 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6458
I20260810 11:48:42.265717 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:45597
--local_ip_for_outbound_sockets=127.20.93.129
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=39135
--webserver_interface=127.20.93.129
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:48:42.265882 6531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap starting.
I20260810 11:48:42.291633 6531 log.cc:824] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Log is configured to *not* fsync() on all Append() calls
I20260810 11:48:42.315864 6531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:48:42.316653 6531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Bootstrap complete.
I20260810 11:48:42.330922 6531 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:42.332284 6531 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Initialized, Role: FOLLOWER
I20260810 11:48:42.333129 6531 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:42.333453 6531 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:48:42.333631 6531 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:48:42.333840 6531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 2 FOLLOWER]: Advancing to term 3
I20260810 11:48:42.340432 6531 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:42.341135 6531 leader_election.cc:302] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc5bc3b4b20a4e509f2ffa928eb8ec12; no voters:
I20260810 11:48:42.342037 6531 leader_election.cc:288] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 11:48:42.342216 6535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 11:48:42.343705 6535 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [term 3 LEADER]: Becoming Leader. State: Replica: dc5bc3b4b20a4e509f2ffa928eb8ec12, State: Running, Role: LEADER
I20260810 11:48:42.344378 6535 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } }
I20260810 11:48:42.344471 6531 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:48:42.347579 6537 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc5bc3b4b20a4e509f2ffa928eb8ec12. Latest consensus state: current_term: 3 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:48:42.347566 6536 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc5bc3b4b20a4e509f2ffa928eb8ec12" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44307 } } }
I20260810 11:48:42.348094 6537 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:48:42.348122 6536 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12 [sys.catalog]: This master's current role is: LEADER
I20260810 11:48:42.360764 6541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:48:42.370111 6541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:48:42.371816 6541 catalog_manager.cc:1259] Loaded cluster ID: a26a9e0647c94d9c85b47b6b5096e611
I20260810 11:48:42.371953 6541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:48:42.376389 6541 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:48:42.378172 6541 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Loaded TSK: 0
I20260810 11:48:42.378341 6541 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc5bc3b4b20a4e509f2ffa928eb8ec12: Loaded TSK: 1
F20260810 11:48:42.379592 6541 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f595cdb29ad google::LogMessage::Fail() at ??:0
@ 0x7f595cdb588a google::LogMessage::SendToLog() at ??:0
@ 0x7f595cdb24de google::LogMessage::Flush() at ??:0
@ 0x7f595cdb3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5960b167d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5960b0573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f59727e11c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f59727e0783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
I20260810 11:48:42.426169 6558 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f597286c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f597286c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f59728b65e9 std::function<>::operator()() at ??:0
@ 0x7f59727e4cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f59727e2c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f597286a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f597286a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f595f9fe345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f595fa0191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f595fa01799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f595f9dceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5962cb66db start_thread at ??:0
@ 0x7f595bc4c61f clone at ??:0
W20260810 11:48:42.729104 6533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:42.729485 6533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:42.729738 6533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:42.738346 6533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:42.738531 6533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:42.738749 6533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:48:42.749740 6533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:45597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=39135
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:42.751304 6533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:42.752729 6533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:42.767055 6584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:42.767127 6583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:42.767103 6587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:42.768641 6533 server_base.cc:1034] running on GCE node
I20260810 11:48:42.769383 6533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:42.770519 6533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:42.771786 6533 hybrid_clock.cc:648] HybridClock initialized: now 1786362522771740 us; error 68 us; skew 500 ppm
I20260810 11:48:42.772153 6533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:42 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:42.775650 6533 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:48:42.777326 6533 webserver.cc:460] Webserver started at http://127.20.93.129:39135/ using document root <none> and password file <none>
I20260810 11:48:42.778165 6533 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:42.778275 6533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:42.784058 6533 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 11:48:42.787560 6595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:42.789000 6533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:48:42.789191 6533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "18927438e2414942b95a81694f423065"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "f61d5190d4b6ad91b1b25e45bb8d0650"
server_key_iv: "659c8ca7d5096d4d3348c7f2ee08e1cf"
server_key_version: "encryptionkey@0"
I20260810 11:48:42.790279 6533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:42.818279 6533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:42.822780 6533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:42.823110 6533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:42.824560 6533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:42.826948 6603 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260810 11:48:42.832738 6603 ts_tablet_manager.cc:1852] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 11:48:42.833879 6603 ts_tablet_manager.cc:1905] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:48:42.841508 6603 ts_tablet_manager.cc:1918] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:48:42.841816 6603 log.cc:1196] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/42092d1a2b374f0d9568f8ed596b5de9
I20260810 11:48:42.842381 6603 ts_tablet_manager.cc:1939] T 42092d1a2b374f0d9568f8ed596b5de9 P 18927438e2414942b95a81694f423065: Deleting consensus metadata
I20260810 11:48:42.842829 6533 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 11:48:42.842983 6533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260810 11:48:42.843150 6533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:42.843271 6533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:42.880390 6533 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:45597
I20260810 11:48:42.880450 6719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:45597 every 8 connection(s)
I20260810 11:48:42.881800 6533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:48:42.886991 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:42.884859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:48547)
0810 11:48:42.885633 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:42.885976 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":272,"threads_started":1}
W20260810 11:48:42.887770 6720 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:44307 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:44307: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
I20260810 11:48:42.887826 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6533
I20260810 11:48:42.888501 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:43499
--local_ip_for_outbound_sockets=127.20.93.130
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=37487
--webserver_interface=127.20.93.130
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:48:42.888758 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:42.888329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:60357)
0810 11:48:42.888491 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:42.888594 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 11:48:42.889783 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:42.889282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:44163)
0810 11:48:42.889475 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:42.889610 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:48:42.890058 6720 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:48:43.255193 6724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:43.255514 6724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:43.255625 6724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:43.263724 6724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:43.263881 6724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:43.264004 6724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:48:43.274447 6724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:43499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=37487
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:43.275982 6724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:43.277415 6724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:43.291596 6734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:43.291682 6729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:43.291705 6730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:43.292604 6724 server_base.cc:1034] running on GCE node
I20260810 11:48:43.293325 6724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:43.294449 6724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:43.295719 6724 hybrid_clock.cc:648] HybridClock initialized: now 1786362523295639 us; error 71 us; skew 500 ppm
I20260810 11:48:43.296075 6724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:43 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:43.299464 6724 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:48:43.301167 6724 webserver.cc:460] Webserver started at http://127.20.93.130:37487/ using document root <none> and password file <none>
I20260810 11:48:43.302021 6724 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:43.302182 6724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:43.307957 6724 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 11:48:43.310954 6741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.312556 6724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:48:43.312769 6724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "754df297a61b4df8b88d037b82da7d3b"
format_stamp: "Formatted at 2026-08-10 11:47:56 on dist-test-slave-njxd"
server_key: "b8d6ee1b73b0f16b6a833abd4a9e4e01"
server_key_iv: "83803aa76c983bac297322bcfd7a2487"
server_key_version: "encryptionkey@0"
I20260810 11:48:43.313969 6724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:43.346593 6724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:43.351282 6724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:43.351586 6724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:43.352687 6724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:43.354230 6724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:48:43.354355 6724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.354498 6724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:43.354605 6724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.391497 6724 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:43499
I20260810 11:48:43.391613 6866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:43499 every 8 connection(s)
I20260810 11:48:43.393087 6724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:48:43.398515 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.396447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:48745)
0810 11:48:43.397244 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.397512 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":309,"threads_started":1}
W20260810 11:48:43.399209 6867 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:44307 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:44307: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
I20260810 11:48:43.400141 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.399740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:34053)
0810 11:48:43.399910 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.400004 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:48:43.400943 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6724
I20260810 11:48:43.401217 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.400773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:44543)
0810 11:48:43.400939 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.401075 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:48:43.401494 6867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 11:48:43.401789 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:34967
--local_ip_for_outbound_sockets=127.20.93.131
--tserver_master_addrs=127.20.93.190:44307
--webserver_port=37275
--webserver_interface=127.20.93.131
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:48:43.767397 6870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:48:43.767706 6870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:48:43.767868 6870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:48:43.775636 6870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:48:43.775755 6870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:48:43.775956 6870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:48:43.786373 6870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:34967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=37275
--tserver_master_addrs=127.20.93.190:44307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:48:43.787827 6870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:48:43.789211 6870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:48:43.803259 6876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:43.803247 6877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:48:43.803247 6879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:48:43.804261 6870 server_base.cc:1034] running on GCE node
I20260810 11:48:43.804922 6870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:48:43.806181 6870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:48:43.807430 6870 hybrid_clock.cc:648] HybridClock initialized: now 1786362523807374 us; error 41 us; skew 500 ppm
I20260810 11:48:43.807814 6870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:48:43 dist-test-slave-njxd krb5kdc[5222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362523, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:48:43.811462 6870 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:48:43.813109 6870 webserver.cc:460] Webserver started at http://127.20.93.131:37275/ using document root <none> and password file <none>
I20260810 11:48:43.813963 6870 fs_manager.cc:359] Metadata directory not provided
I20260810 11:48:43.814128 6870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:48:43.820575 6870 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 11:48:43.823843 6886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.825407 6870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:48:43.825711 6870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "b016c6e7a08d4d64ba1f589f19cbf7e0"
format_stamp: "Formatted at 2026-08-10 11:47:57 on dist-test-slave-njxd"
server_key: "d19f902b770ed5c05be7f0e1dc78b52a"
server_key_iv: "b5ebbfed75ce36e14916e0bb0e8860f1"
server_key_version: "encryptionkey@0"
I20260810 11:48:43.826936 6870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:48:43.847301 6870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:48:43.852160 6870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:48:43.852461 6870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:48:43.853631 6870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:48:43.855134 6870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:48:43.855283 6870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.855430 6870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:48:43.855522 6870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:48:43.891820 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.890773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:33275)
0810 11:48:43.891320 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.891467 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":273,"threads_started":1}
I20260810 11:48:43.892891 6870 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:34967
I20260810 11:48:43.892939 7014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:34967 every 8 connection(s)
I20260810 11:48:43.894542 6870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:48:43.899822 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.897908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:48919)
0810 11:48:43.898653 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.898915 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":284,"threads_started":1}
W20260810 11:48:43.900581 7015 heartbeater.cc:644] Failed to heartbeat to 127.20.93.190:44307 (0 consecutive failures): Network error: Failed to ping master at 127.20.93.190:44307: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
I20260810 11:48:43.901542 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.901074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:40801)
0810 11:48:43.901341 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.901427 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 11:48:43.902534 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:43.902173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:45073)
0810 11:48:43.902322 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:43.902422 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:48:43.902793 7015 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 11:48:43.903229 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 6870
I20260810 11:48:44.403136 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:44.402163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:35917)
0810 11:48:44.402669 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:44.402805 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":243,"threads_started":1}
I20260810 11:48:44.893610 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:44.892745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:59273)
0810 11:48:44.893247 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:44.893372 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":246,"threads_started":1}
I20260810 11:48:44.904248 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:44.903425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:54139)
0810 11:48:44.903891 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:44.904001 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260810 11:48:45.404781 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:45.404055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:44459)
0810 11:48:45.404527 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:45.404610 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":223,"threads_started":1}
I20260810 11:48:45.895529 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:45.894623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:58535)
0810 11:48:45.895241 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:45.895335 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":358,"threads_started":1}
I20260810 11:48:45.906106 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:45.905290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:52199)
0810 11:48:45.905813 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:45.905915 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":259,"threads_started":1}
I20260810 11:48:46.406721 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:46.405898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:52821)
0810 11:48:46.406441 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:46.406535 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":298,"threads_started":1}
I20260810 11:48:46.897531 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:46.896635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:55081)
0810 11:48:46.897200 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:46.897320 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":290,"threads_started":1}
I20260810 11:48:46.907879 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:46.907145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:46869)
0810 11:48:46.907615 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:46.907709 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":254,"threads_started":1}
I20260810 11:48:47.408735 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:47.407821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:49397)
0810 11:48:47.408446 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:47.408542 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":260,"threads_started":1}
I20260810 11:48:47.899533 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:47.898683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:58283)
0810 11:48:47.899267 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:47.899359 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":341,"threads_started":1}
I20260810 11:48:47.909904 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:47.908993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:46805)
0810 11:48:47.909568 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:47.909717 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":316,"threads_started":1}
I20260810 11:48:48.410693 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:48.409865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:32899)
0810 11:48:48.410407 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:48.410501 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":299,"threads_started":1}
I20260810 11:48:48.901335 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:48.900516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:51505)
0810 11:48:48.901001 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:48.901092 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":236,"threads_started":1}
I20260810 11:48:48.911830 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:48.910883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:57375)
0810 11:48:48.911484 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:48.911623 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":282,"threads_started":1}
I20260810 11:48:49.412895 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:49.411856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:60815)
0810 11:48:49.412518 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:49.412672 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":329,"threads_started":1}
I20260810 11:48:49.903174 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:49.902359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:58589)
0810 11:48:49.902883 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:49.902970 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":252,"threads_started":1}
I20260810 11:48:49.913889 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:49.912837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:49869)
0810 11:48:49.913502 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:49.913695 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":316,"threads_started":1}
I20260810 11:48:50.415499 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:50.414122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:32953)
0810 11:48:50.415176 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:50.415272 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":939,"thread_start_us":719,"threads_started":1}
I20260810 11:48:50.905112 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:50.904308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:42353)
0810 11:48:50.904798 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:50.904885 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":287,"threads_started":1}
I20260810 11:48:50.916121 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:50.915093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:38229)
0810 11:48:50.915687 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:50.915835 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":266,"threads_started":1}
I20260810 11:48:51.417412 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:51.416598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:44583)
0810 11:48:51.417149 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:51.417243 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":224,"threads_started":1}
I20260810 11:48:51.907068 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:51.906239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:34673)
0810 11:48:51.906765 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:51.906876 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":246,"threads_started":1}
I20260810 11:48:51.918267 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:51.917232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:45979)
0810 11:48:51.917905 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:51.918004 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":417,"threads_started":1}
I20260810 11:48:52.419395 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:52.418447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:60383)
0810 11:48:52.419095 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:52.419215 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":336,"threads_started":1}
I20260810 11:48:52.908849 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:52.908098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:57709)
0810 11:48:52.908567 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:52.908674 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260810 11:48:52.920224 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:52.919301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:47093)
0810 11:48:52.919930 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:52.920036 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":282,"threads_started":1}
I20260810 11:48:53.421259 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:53.420386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:60721)
0810 11:48:53.421001 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:53.421087 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":365,"threads_started":1}
I20260810 11:48:53.910682 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:53.909861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:37627)
0810 11:48:53.910387 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:53.910476 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":315,"threads_started":1}
I20260810 11:48:53.922068 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:53.921227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:35221)
0810 11:48:53.921810 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:53.921897 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":293,"threads_started":1}
I20260810 11:48:54.422962 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:54.422192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:33255)
0810 11:48:54.422662 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:54.422776 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":251,"threads_started":1}
I20260810 11:48:54.912472 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:54.911723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:60703)
0810 11:48:54.912194 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:54.912283 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":250,"threads_started":1}
I20260810 11:48:54.923991 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:54.923131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:53903)
0810 11:48:54.923691 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:54.923790 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":319,"threads_started":1}
I20260810 11:48:55.424787 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:55.423992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:49315)
0810 11:48:55.424517 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:55.424617 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":262,"threads_started":1}
I20260810 11:48:55.914434 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:55.913550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:48141)
0810 11:48:55.914125 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:55.914234 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":263,"threads_started":1}
I20260810 11:48:55.925894 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:55.925087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:32827)
0810 11:48:55.925632 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:55.925726 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260810 11:48:56.426847 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:56.426008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:60117)
0810 11:48:56.426544 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:56.426646 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":273,"threads_started":1}
I20260810 11:48:56.916240 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:56.915452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:35531)
0810 11:48:56.915936 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:56.916036 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":246,"threads_started":1}
I20260810 11:48:56.927747 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:56.926827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:52609)
0810 11:48:56.927428 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:56.927527 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":335,"threads_started":1}
I20260810 11:48:57.428786 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:57.427916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:55525)
0810 11:48:57.428493 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:57.428596 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":328,"threads_started":1}
I20260810 11:48:57.918766 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:57.917781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:41425)
0810 11:48:57.918414 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:57.918534 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":285,"threads_started":1}
I20260810 11:48:57.929683 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:57.928746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:58133)
0810 11:48:57.929326 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:57.929418 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":327,"threads_started":1}
I20260810 11:48:58.430935 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:58.430029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.130:39371)
0810 11:48:58.430603 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:58.430732 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":270,"threads_started":1}
I20260810 11:48:58.911127 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 6533
I20260810 11:48:58.920929 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:58.919794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.129:58297)
0810 11:48:58.920449 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:58.920657 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":286,"threads_started":1}
I20260810 11:48:58.932195 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:58.931217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44307 (local address 127.20.93.131:48429)
0810 11:48:58.931808 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 11:48:58.931909 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.190:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":342,"threads_started":1}
I20260810 11:48:58.938638 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:58.912384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.20.93.129:45597)
0810 11:48:58.913028 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:58.913043 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:58.913088 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:48:58.914816 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:58.914851 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:58.915327 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:58.915649 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:58.917193 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:58.918748 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:58.920565 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:58.921676 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:58.921860 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:58.930499 (+ 8639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:58.930516 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:58.930538 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:58.930602 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:58.934845 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:58.935802 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:58.935811 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:58.935820 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:58.936018 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:58.936464 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:58.936467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:58.936475 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:58.937099 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:58.937344 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:58.937948 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 11:48:58.938322 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":323,"threads_started":1}
I20260810 11:48:58.940512 6715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:59.030867 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 6533
I20260810 11:48:59.052847 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 6724
I20260810 11:48:59.073098 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:59.053978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.20.93.130:43499)
0810 11:48:59.054544 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:59.054559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:59.054597 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:48:59.056206 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:59.056217 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:59.056707 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:59.057112 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:59.058864 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:59.060159 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:59.061627 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:59.062484 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:59.062722 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:59.065716 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:59.065727 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:59.065746 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:59.065794 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:59.069116 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:59.070098 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:59.070107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:59.070115 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:59.070317 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:59.070778 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:59.070787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:59.070789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:59.071386 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:59.071732 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:59.072313 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 11:48:59.072790 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":262,"threads_started":1}
I20260810 11:48:59.075289 6862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:59.162278 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 6724
I20260810 11:48:59.183076 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 6870
I20260810 11:48:59.203860 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:48:59.184257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.20.93.131:34967)
0810 11:48:59.184948 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 11:48:59.184963 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:48:59.185009 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:48:59.186949 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:48:59.186959 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:48:59.187421 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:48:59.187759 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:48:59.189312 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:59.190801 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:59.192227 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:48:59.193008 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:48:59.193171 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:48:59.196205 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:48:59.196215 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:48:59.196235 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:48:59.196279 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:48:59.199735 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:59.200577 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:59.200585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:59.200594 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:59.200799 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:48:59.201315 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:48:59.201320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:48:59.201322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:48:59.202343 (+ 1021us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:48:59.202601 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 11:48:59.203102 (+ 501us) server_negotiation.cc:299] Negotiation successful
0810 11:48:59.203497 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":319,"threads_started":1}
I20260810 11:48:59.205490 7009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:48:59.291968 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 6870
2026-08-10T11:48:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65861 ms)
[----------] 30 tests from SecurityITest (310471 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:48:59 dist-test-slave-njxd krb5kdc[7080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:48:59 dist-test-slave-njxd krb5kdc[7080](info): set up 2 sockets
Aug 10 11:48:59 dist-test-slave-njxd krb5kdc[7080](info): commencing operation
krb5kdc: starting...
W20260810 11:49:00.513365 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:00 dist-test-slave-njxd krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:00.837553 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33371
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:33371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:01.192682 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:01.192984 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:01.193141 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:01.200589 7094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:01.200701 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:01.200788 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:01.200899 7094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:01.200966 7094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:01.210898 7094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:33371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:33371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:01.212297 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:01.213786 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:01.226012 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:01.226073 7102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:01.226106 7100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:01.226907 7094 server_base.cc:1034] running on GCE node
I20260810 11:49:01.227653 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:01.229063 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:01.230270 7094 hybrid_clock.cc:648] HybridClock initialized: now 1786362541230221 us; error 33 us; skew 500 ppm
I20260810 11:49:01.230614 7094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:01 dist-test-slave-njxd krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:01.233870 7094 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:01.235401 7094 webserver.cc:460] Webserver started at http://127.20.93.190:37929/ using document root <none> and password file <none>
I20260810 11:49:01.236111 7094 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:01.236264 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:01.236554 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:01.239729 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a1ec28328975470cb8c0b7b6aaf331f2"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "eafa7965d3aeed7294f12cf357196c82"
server_key_iv: "cf2d673cbc4bc22a28ca68cd9ef0fa6e"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.240454 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a1ec28328975470cb8c0b7b6aaf331f2"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "eafa7965d3aeed7294f12cf357196c82"
server_key_iv: "cf2d673cbc4bc22a28ca68cd9ef0fa6e"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.245873 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 11:49:01.249365 7110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:01.250900 7094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:01.251102 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a1ec28328975470cb8c0b7b6aaf331f2"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "eafa7965d3aeed7294f12cf357196c82"
server_key_iv: "cf2d673cbc4bc22a28ca68cd9ef0fa6e"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.251322 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:01.268682 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:01.272949 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:01.273286 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:01.293112 7094 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:33371
I20260810 11:49:01.293131 7167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:33371 every 8 connection(s)
I20260810 11:49:01.294651 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:01.298643 7168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:01.298899 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7094
I20260810 11:49:01.299139 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:01.300016 20854 external_mini_cluster.cc:1428] Setting key c0d0534ff984c758bedb06d97d3346a8
I20260810 11:49:01.314313 7168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: Bootstrap starting.
Aug 10 11:49:01 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:01.318280 7168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:01.319516 7168 log.cc:824] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:01.322820 7168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: No bootstrap required, opened a new log
I20260810 11:49:01.326481 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:01.302324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.20.93.190:33371)
0810 11:49:01.303532 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:01.303544 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:01.303586 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:49:01.306108 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:01.306134 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:01.306585 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:01.306943 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:01.309046 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.310518 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.312376 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.313052 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.313203 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:01.318211 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:01.318223 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:01.318238 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:01.318290 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:01.322014 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:01.322825 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:01.322841 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:01.322848 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:01.323082 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:01.323521 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:01.323526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:01.323530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:01.324074 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:01.324416 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:01.324901 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 11:49:01.325344 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":983,"thread_start_us":695,"threads_started":1}
I20260810 11:49:01.331494 7168 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } }
I20260810 11:49:01.331893 7168 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:01.332001 7168 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ec28328975470cb8c0b7b6aaf331f2, State: Initialized, Role: FOLLOWER
I20260810 11:49:01.332558 7168 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } }
I20260810 11:49:01.332813 7168 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:01.332993 7168 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:01.333171 7168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:01.335942 7168 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } }
I20260810 11:49:01.336509 7168 leader_election.cc:302] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1ec28328975470cb8c0b7b6aaf331f2; no voters:
I20260810 11:49:01.337121 7168 leader_election.cc:288] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:01.337255 7173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:01.338799 7173 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [term 1 LEADER]: Becoming Leader. State: Replica: a1ec28328975470cb8c0b7b6aaf331f2, State: Running, Role: LEADER
I20260810 11:49:01.339282 7168 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:01.339319 7173 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } }
I20260810 11:49:01.342929 7175 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ec28328975470cb8c0b7b6aaf331f2. Latest consensus state: current_term: 1 leader_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } } }
I20260810 11:49:01.342955 7174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ec28328975470cb8c0b7b6aaf331f2" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 33371 } } }
I20260810 11:49:01.343381 7175 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:01.343381 7174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:01.346119 7184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:01.352061 7184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:01.361380 7184 catalog_manager.cc:1347] Generated new cluster ID: 1b4b00aa2ca34477a8d2ce099d33bfcb
I20260810 11:49:01.361509 7184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:01.371723 7184 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:01.373365 7184 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:01.384176 7184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: Generated new TSK 0
I20260810 11:49:01.385049 7184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:49:01.460731 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:01.834159 7195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:01.834462 7195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:01.834650 7195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:01.842515 7195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:01.842658 7195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:01.842854 7195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:01.853372 7195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:01.854868 7195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:01.856393 7195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:01.870661 7203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:01.870781 7200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:01.870745 7201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:01.871716 7195 server_base.cc:1034] running on GCE node
I20260810 11:49:01.872412 7195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:01.873497 7195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:01.874789 7195 hybrid_clock.cc:648] HybridClock initialized: now 1786362541874711 us; error 63 us; skew 500 ppm
I20260810 11:49:01.875161 7195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:01 dist-test-slave-njxd krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:01.878836 7195 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:49:01.880714 7195 webserver.cc:460] Webserver started at http://127.20.93.129:43511/ using document root <none> and password file <none>
I20260810 11:49:01.881516 7195 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:01.881716 7195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:01.882081 7195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:01.885528 7195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "92d41c1694404fcca967810240f07216"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "6d4c3cc3b3bb16acfe0a21c6ed670f20"
server_key_iv: "47781652323b05431a8b2c8efea77374"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.886469 7195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "92d41c1694404fcca967810240f07216"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "6d4c3cc3b3bb16acfe0a21c6ed670f20"
server_key_iv: "47781652323b05431a8b2c8efea77374"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.892395 7195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:01.896302 7210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:01.897969 7195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:01.898183 7195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "92d41c1694404fcca967810240f07216"
format_stamp: "Formatted at 2026-08-10 11:49:01 on dist-test-slave-njxd"
server_key: "6d4c3cc3b3bb16acfe0a21c6ed670f20"
server_key_iv: "47781652323b05431a8b2c8efea77374"
server_key_version: "encryptionkey@0"
I20260810 11:49:01.898416 7195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:01.918084 7195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:01.922497 7195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:01.922843 7195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:01.923907 7195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:01.925401 7195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:01.925513 7195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:01.925704 7195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:01.925782 7195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:01.961453 7195 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:33591
I20260810 11:49:01.961499 7332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:33591 every 8 connection(s)
I20260810 11:49:01.962885 7195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:01.963832 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7195
I20260810 11:49:01.964094 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:01.964926 20854 external_mini_cluster.cc:1428] Setting key 476616e999913c86d4200becc74d250a
Aug 10 11:49:01 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:01.988700 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:01.966119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:46743 (local address 127.20.93.190:33371)
0810 11:49:01.966677 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:01.966684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:01.967821 (+ 1137us) server_negotiation.cc:407] Connection header received
0810 11:49:01.970899 (+ 3078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:01.970904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:01.970994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:01.971094 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:01.973453 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.974313 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.976934 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.977378 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.977499 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:01.982219 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:01.982225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:01.982229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:01.982273 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:01.985092 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:01.986089 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:01.986096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:01.986102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:01.986243 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:01.986750 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:01.986755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:01.986758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:01.987041 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:01.987220 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:01.988021 (+ 801us) server_negotiation.cc:299] Negotiation successful
0810 11:49:01.988303 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":290,"threads_started":1}
I20260810 11:49:01.989953 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:01.966756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33371 (local address 127.20.93.129:46743)
0810 11:49:01.967626 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:01.967687 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:01.970554 (+ 2867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:01.971405 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:01.971421 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:01.972327 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:01.973216 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:01.973248 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.974487 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.974495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:01.976760 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:01.976780 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:01.977672 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:01.977683 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:01.978029 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:01.978883 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:01.978924 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:01.981914 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:01.985291 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:01.985302 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:01.985323 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:01.985931 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:01.986371 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:01.986376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:01.986380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:01.986607 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:01.987195 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:01.987205 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:01.987608 (+ 403us) client_negotiation.cc:769] Sending connection context
0810 11:49:01.987903 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 11:49:01.988519 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":334,"threads_started":1}
I20260810 11:49:01.991920 7333 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33371
I20260810 11:49:01.992313 7333 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:01.993309 7333 heartbeater.cc:511] Master 127.20.93.190:33371 requested a full tablet report, sending...
I20260810 11:49:01.996215 7127 ts_manager.cc:194] Registered new tserver with Master: 92d41c1694404fcca967810240f07216 (127.20.93.129:33591)
I20260810 11:49:01.998651 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:46743
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:49:02.046749 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:02.398811 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:02.399119 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:02.399223 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:02.406836 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:02.406961 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:02.407078 7339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:02.417187 7339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:02.418668 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:02.420029 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:02.434518 7345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:02.434504 7347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:02.434945 7344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:02.435670 7339 server_base.cc:1034] running on GCE node
I20260810 11:49:02.436403 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:02.437647 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:02.439002 7339 hybrid_clock.cc:648] HybridClock initialized: now 1786362542438949 us; error 44 us; skew 500 ppm
I20260810 11:49:02.439459 7339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:02 dist-test-slave-njxd krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362542, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:02.443235 7339 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:49:02.444976 7339 webserver.cc:460] Webserver started at http://127.20.93.130:33053/ using document root <none> and password file <none>
I20260810 11:49:02.445921 7339 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:02.446076 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:02.446455 7339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:02.449806 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "3175c7c25e7244f396941e84ecf8410d"
format_stamp: "Formatted at 2026-08-10 11:49:02 on dist-test-slave-njxd"
server_key: "8bec5f8d5d263a2c5e30298338df9ad7"
server_key_iv: "01bb2b56e2b90c615f0b7ba86bc27c2c"
server_key_version: "encryptionkey@0"
I20260810 11:49:02.450664 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "3175c7c25e7244f396941e84ecf8410d"
format_stamp: "Formatted at 2026-08-10 11:49:02 on dist-test-slave-njxd"
server_key: "8bec5f8d5d263a2c5e30298338df9ad7"
server_key_iv: "01bb2b56e2b90c615f0b7ba86bc27c2c"
server_key_version: "encryptionkey@0"
I20260810 11:49:02.456620 7339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:49:02.460443 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:02.462085 7339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:02.462283 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "3175c7c25e7244f396941e84ecf8410d"
format_stamp: "Formatted at 2026-08-10 11:49:02 on dist-test-slave-njxd"
server_key: "8bec5f8d5d263a2c5e30298338df9ad7"
server_key_iv: "01bb2b56e2b90c615f0b7ba86bc27c2c"
server_key_version: "encryptionkey@0"
I20260810 11:49:02.462519 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:02.479785 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:02.484488 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:02.484802 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:02.485888 7339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:02.487313 7339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:02.487385 7339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:02.487545 7339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:02.487597 7339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:02.523132 7339 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:38035
I20260810 11:49:02.523186 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:38035 every 8 connection(s)
I20260810 11:49:02.524597 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:02.530273 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7339
I20260810 11:49:02.530498 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:02.530990 20854 external_mini_cluster.cc:1428] Setting key a1c675a7770c1006741a03a912f5b0fd
Aug 10 11:49:02 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362542, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:02.549858 7479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:02.527328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:49523 (local address 127.20.93.190:33371)
0810 11:49:02.527828 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:02.527835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:02.529112 (+ 1277us) server_negotiation.cc:407] Connection header received
0810 11:49:02.531379 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:02.531385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:02.531492 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:02.531640 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:02.533514 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:02.534694 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:02.536400 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:02.537182 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:02.537329 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:02.542340 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:02.542350 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:02.542358 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:02.542408 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:02.546382 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:02.547237 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:02.547245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:02.547251 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:02.547427 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:02.547928 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:02.547935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:02.547938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:02.548247 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:02.548419 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:02.549181 (+ 762us) server_negotiation.cc:299] Negotiation successful
0810 11:49:02.549432 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260810 11:49:02.550949 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:02.527970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33371 (local address 127.20.93.130:49523)
0810 11:49:02.528884 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:02.528943 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:02.531005 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:02.531989 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:02.531998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:02.532558 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:02.533331 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:02.533351 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:02.534891 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:02.534895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:02.536233 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:02.536257 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:02.537445 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:02.537454 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:02.537817 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:02.538642 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:02.538677 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:02.541999 (+ 3322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:02.546569 (+ 4570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:02.546577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:02.546608 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:02.547097 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:02.547578 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:02.547583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:02.547586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:02.547804 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:02.548393 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:02.548403 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:02.548796 (+ 393us) client_negotiation.cc:769] Sending connection context
0810 11:49:02.549100 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 11:49:02.549746 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":382,"threads_started":1}
I20260810 11:49:02.552491 7477 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33371
I20260810 11:49:02.552847 7477 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:02.553833 7477 heartbeater.cc:511] Master 127.20.93.190:33371 requested a full tablet report, sending...
I20260810 11:49:02.555702 7127 ts_manager.cc:194] Registered new tserver with Master: 3175c7c25e7244f396941e84ecf8410d (127.20.93.130:38035)
I20260810 11:49:02.556651 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:49523
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:49:02.605420 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:38841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:02.958199 7483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:02.958489 7483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:02.958679 7483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:02.966464 7483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:02.966601 7483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:02.966774 7483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:02.977029 7483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:38841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:02.978439 7483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:02.979784 7483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:02.993810 7489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:02.994066 7490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:02.993965 7492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:02.995134 7483 server_base.cc:1034] running on GCE node
I20260810 11:49:02.995783 7483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:02.996774 7483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:02.998006 7483 hybrid_clock.cc:648] HybridClock initialized: now 1786362542997937 us; error 59 us; skew 500 ppm
I20260810 11:49:02.998339 7483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:02 dist-test-slave-njxd krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362542, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:03.001971 7483 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:49:03.003028 7333 heartbeater.cc:503] Master 127.20.93.190:33371 was elected leader, sending a full tablet report...
I20260810 11:49:03.003741 7483 webserver.cc:460] Webserver started at http://127.20.93.131:35043/ using document root <none> and password file <none>
I20260810 11:49:03.004468 7483 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:03.004577 7483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:03.004900 7483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:03.007967 7483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "b8aa1690053f41579f1f3c8f96a46d7c"
format_stamp: "Formatted at 2026-08-10 11:49:03 on dist-test-slave-njxd"
server_key: "d6ece576944b29a74207f238b50ef94a"
server_key_iv: "5ea6ab0998cfee9b5df43a01a044e79d"
server_key_version: "encryptionkey@0"
I20260810 11:49:03.008802 7483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "b8aa1690053f41579f1f3c8f96a46d7c"
format_stamp: "Formatted at 2026-08-10 11:49:03 on dist-test-slave-njxd"
server_key: "d6ece576944b29a74207f238b50ef94a"
server_key_iv: "5ea6ab0998cfee9b5df43a01a044e79d"
server_key_version: "encryptionkey@0"
I20260810 11:49:03.015559 7483 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 11:49:03.019304 7500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:03.020700 7483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:49:03.020833 7483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "b8aa1690053f41579f1f3c8f96a46d7c"
format_stamp: "Formatted at 2026-08-10 11:49:03 on dist-test-slave-njxd"
server_key: "d6ece576944b29a74207f238b50ef94a"
server_key_iv: "5ea6ab0998cfee9b5df43a01a044e79d"
server_key_version: "encryptionkey@0"
I20260810 11:49:03.021010 7483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:03.033264 7483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:03.037943 7483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:03.038206 7483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:03.039239 7483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:03.040863 7483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:03.040939 7483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:03.041023 7483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:03.041069 7483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:03.076502 7483 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:43485
I20260810 11:49:03.076538 7621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:43485 every 8 connection(s)
I20260810 11:49:03.078164 7483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:03.087811 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7483
I20260810 11:49:03.088006 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:03.088543 20854 external_mini_cluster.cc:1428] Setting key fcc6cf5cbe61038d682dd8129f24d360
Aug 10 11:49:03 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362542, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:03.098656 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.080858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:53541 (local address 127.20.93.190:33371)
0810 11:49:03.081380 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.081387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.082470 (+ 1083us) server_negotiation.cc:407] Connection header received
0810 11:49:03.084264 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.084268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.084349 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.084445 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:03.086065 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.086885 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.088546 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.089170 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.089278 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.093138 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:03.093143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:03.093147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:03.093187 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:03.095791 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.096534 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.096538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.096541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.096666 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.097069 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.097072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.097074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.097343 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:03.097456 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.098207 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.098377 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
I20260810 11:49:03.099660 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.081476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:33371 (local address 127.20.93.131:53541)
0810 11:49:03.082293 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.082345 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.084005 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.084654 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.084662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.085199 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:03.085889 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.085907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.087017 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.087020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.088388 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.088405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.089418 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.089428 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.089757 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.090626 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:03.090658 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:03.092934 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:03.095925 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:03.095933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:03.095947 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:03.096422 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:03.096780 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:03.096783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:03.096785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:03.096967 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:03.097447 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:03.097452 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:03.097858 (+ 406us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.098097 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.098511 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":285,"threads_started":1}
I20260810 11:49:03.101101 7622 heartbeater.cc:347] Connected to a master server at 127.20.93.190:33371
I20260810 11:49:03.101410 7622 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:03.102243 7622 heartbeater.cc:511] Master 127.20.93.190:33371 requested a full tablet report, sending...
I20260810 11:49:03.103813 7127 ts_manager.cc:194] Registered new tserver with Master: b8aa1690053f41579f1f3c8f96a46d7c (127.20.93.131:43485)
I20260810 11:49:03.104456 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:03.104699 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:53541
I20260810 11:49:03.121376 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.109035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.20.93.190:33371)
0810 11:49:03.109260 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.109269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.109681 (+ 412us) server_negotiation.cc:407] Connection header received
0810 11:49:03.109845 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.109849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.109910 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.109987 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:03.111367 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.112045 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.113376 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.113832 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.113929 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.116475 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:03.116480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:03.116484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:03.116514 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:03.118949 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.119603 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.119607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.119609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.119741 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.120057 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.120060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.120062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.120385 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:03.120565 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.120858 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.121026 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:49:03.125416 7127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:03.132318 7127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:03.156128 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.145341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.20.93.129:33591)
0810 11:49:03.147635 (+ 2294us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.147646 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.147679 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:49:03.147790 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.147800 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.148070 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.148305 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:03.149024 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.150658 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.153452 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.154826 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.154970 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.155209 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.155602 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.155786 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2114,"thread_start_us":1952,"threads_started":1}
I20260810 11:49:03.156167 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.145035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:33591 (local address 127.0.0.1:36658)
0810 11:49:03.146653 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.146713 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.146921 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.148305 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.148312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.148347 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:03.148795 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.148810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.150847 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.150854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.153253 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.153267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.155039 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.155045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.155337 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.155397 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.155536 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.155710 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":3,"thread_start_us":5772,"threads_started":1}
I20260810 11:49:03.157747 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.145044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:38035 (local address 127.0.0.1:50864)
0810 11:49:03.149001 (+ 3957us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.149019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.149221 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.150102 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.150106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.150133 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:03.150543 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.150554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.152471 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.152479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.154469 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.154482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.156853 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.156859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.157017 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.157047 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.157151 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.157287 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3820,"mutex_wait_us":302,"thread_start_us":866,"threads_started":1}
I20260810 11:49:03.157752 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.145246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.20.93.130:38035)
0810 11:49:03.145958 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.145971 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.149111 (+ 3140us) server_negotiation.cc:407] Connection header received
0810 11:49:03.149439 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.149447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.149684 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.149845 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:03.150751 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.152295 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.154666 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.156081 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.156237 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.156703 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.157227 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.157387 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":317,"threads_started":1}
I20260810 11:49:03.159888 7261 tablet_service.cc:1467] Processing CreateTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c (DEFAULT_TABLE table=test-table [id=77dcf9487ac34e2f87afb50c75e55730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:03.162199 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.146908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:43485 (local address 127.0.0.1:51468)
0810 11:49:03.151143 (+ 4235us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.151161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.151296 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.151869 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.151875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.151908 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:03.152285 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.152296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.154217 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.154222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.156679 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.156692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.161321 (+ 4629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.161328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.161503 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.161541 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.161705 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.161853 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4098,"thread_start_us":1805,"threads_started":1}
I20260810 11:49:03.162603 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a75d6d5b566e4b1cabb2bc989833a43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:03.162635 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.147276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.20.93.131:43485)
0810 11:49:03.148046 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.148053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.151272 (+ 3219us) server_negotiation.cc:407] Connection header received
0810 11:49:03.151440 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.151447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.151646 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.151847 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:03.152479 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.154057 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.156829 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.161113 (+ 4284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.161288 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.161511 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.161751 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.161914 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":432,"threads_started":1}
I20260810 11:49:03.164068 7406 tablet_service.cc:1467] Processing CreateTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c (DEFAULT_TABLE table=test-table [id=77dcf9487ac34e2f87afb50c75e55730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:03.165535 7406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a75d6d5b566e4b1cabb2bc989833a43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:03.165678 7551 tablet_service.cc:1467] Processing CreateTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c (DEFAULT_TABLE table=test-table [id=77dcf9487ac34e2f87afb50c75e55730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:03.166962 7551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a75d6d5b566e4b1cabb2bc989833a43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:03.177868 7642 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Bootstrap starting.
I20260810 11:49:03.180645 7644 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Bootstrap starting.
I20260810 11:49:03.181921 7642 tablet_bootstrap.cc:654] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:03.183318 7642 log.cc:824] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:03.183754 7643 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Bootstrap starting.
I20260810 11:49:03.184346 7644 tablet_bootstrap.cc:654] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:03.185634 7644 log.cc:824] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:03.186965 7642 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: No bootstrap required, opened a new log
I20260810 11:49:03.187345 7642 ts_tablet_manager.cc:1397] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 11:49:03.187398 7643 tablet_bootstrap.cc:654] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:03.188652 7643 log.cc:824] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:03.188975 7644 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: No bootstrap required, opened a new log
I20260810 11:49:03.189286 7644 ts_tablet_manager.cc:1397] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 11:49:03.191843 7643 tablet_bootstrap.cc:492] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: No bootstrap required, opened a new log
I20260810 11:49:03.192111 7643 ts_tablet_manager.cc:1397] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 11:49:03.197186 7644 raft_consensus.cc:348] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.197573 7644 raft_consensus.cc:374] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:03.197731 7644 raft_consensus.cc:729] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3175c7c25e7244f396941e84ecf8410d, State: Initialized, Role: FOLLOWER
I20260810 11:49:03.198355 7644 consensus_queue.cc:260] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.198262 7642 raft_consensus.cc:348] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.198679 7642 raft_consensus.cc:374] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:03.198781 7642 raft_consensus.cc:729] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8aa1690053f41579f1f3c8f96a46d7c, State: Initialized, Role: FOLLOWER
I20260810 11:49:03.199589 7477 heartbeater.cc:503] Master 127.20.93.190:33371 was elected leader, sending a full tablet report...
I20260810 11:49:03.199468 7642 consensus_queue.cc:260] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.200212 7644 ts_tablet_manager.cc:1428] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260810 11:49:03.201331 7622 heartbeater.cc:503] Master 127.20.93.190:33371 was elected leader, sending a full tablet report...
I20260810 11:49:03.201926 7642 ts_tablet_manager.cc:1428] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 11:49:03.203033 7643 raft_consensus.cc:348] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.203469 7643 raft_consensus.cc:374] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:03.203583 7643 raft_consensus.cc:729] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d41c1694404fcca967810240f07216, State: Initialized, Role: FOLLOWER
I20260810 11:49:03.204166 7643 consensus_queue.cc:260] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.206004 7643 ts_tablet_manager.cc:1428] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260810 11:49:03.215613 7334 tablet.cc:2369] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:49:03.276805 7478 tablet.cc:2369] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:49:03.330037 7623 tablet.cc:2369] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:03.401471 7649 raft_consensus.cc:482] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:03.401793 7649 raft_consensus.cc:504] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.403669 7649 leader_election.cc:288] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d41c1694404fcca967810240f07216 (127.20.93.129:33591), b8aa1690053f41579f1f3c8f96a46d7c (127.20.93.131:43485)
I20260810 11:49:03.413695 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.404244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:57189 (local address 127.20.93.129:33591)
0810 11:49:03.404476 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.404483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.405315 (+ 832us) server_negotiation.cc:407] Connection header received
0810 11:49:03.405566 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.405572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.405674 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.405782 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:03.406816 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.407971 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.411179 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.412442 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.412618 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.412697 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.413269 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.413407 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 11:49:03.413713 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.404247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:58577 (local address 127.20.93.131:43485)
0810 11:49:03.404476 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.404483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.405385 (+ 902us) server_negotiation.cc:407] Connection header received
0810 11:49:03.405566 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.405602 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.405691 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.405782 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:03.406806 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.407987 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.411172 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.412446 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.412618 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.412715 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.413269 (+ 554us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.413407 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 11:49:03.413910 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.403986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:33591 (local address 127.20.93.130:57189)
0810 11:49:03.405180 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.405203 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.405388 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.405930 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.405936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.405971 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:03.406579 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.406601 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.408137 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.408142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.410985 (+ 2843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.411002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.412789 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.412794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.413060 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.413128 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.413227 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.413409 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"mutex_wait_us":200,"thread_start_us":738,"threads_started":1}
I20260810 11:49:03.414407 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.404059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:43485 (local address 127.20.93.130:58577)
0810 11:49:03.405259 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:03.405277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:03.405424 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:03.405930 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:03.405936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:03.405982 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:03.406612 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.406621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.408136 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.408142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.410985 (+ 2843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:03.411002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.412787 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.412794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:03.413061 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.413118 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 11:49:03.413227 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 11:49:03.413409 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":318,"spinlock_wait_cycles":60928,"thread_start_us":634,"threads_started":1}
I20260810 11:49:03.415163 7282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a75d6d5b566e4b1cabb2bc989833a43c" candidate_uuid: "3175c7c25e7244f396941e84ecf8410d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d41c1694404fcca967810240f07216" is_pre_election: true
I20260810 11:49:03.415403 7573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a75d6d5b566e4b1cabb2bc989833a43c" candidate_uuid: "3175c7c25e7244f396941e84ecf8410d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" is_pre_election: true
I20260810 11:49:03.415752 7282 raft_consensus.cc:2454] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3175c7c25e7244f396941e84ecf8410d in term 0.
I20260810 11:49:03.415944 7573 raft_consensus.cc:2454] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3175c7c25e7244f396941e84ecf8410d in term 0.
I20260810 11:49:03.416507 7356 leader_election.cc:302] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3175c7c25e7244f396941e84ecf8410d, 92d41c1694404fcca967810240f07216; no voters:
I20260810 11:49:03.416956 7649 raft_consensus.cc:2781] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:03.417088 7649 raft_consensus.cc:482] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:03.417232 7649 raft_consensus.cc:3037] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:03.419523 7649 raft_consensus.cc:504] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.420157 7649 leader_election.cc:288] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [CANDIDATE]: Term 1 election: Requested vote from peers 92d41c1694404fcca967810240f07216 (127.20.93.129:33591), b8aa1690053f41579f1f3c8f96a46d7c (127.20.93.131:43485)
I20260810 11:49:03.420691 7573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a75d6d5b566e4b1cabb2bc989833a43c" candidate_uuid: "3175c7c25e7244f396941e84ecf8410d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8aa1690053f41579f1f3c8f96a46d7c"
I20260810 11:49:03.420707 7282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a75d6d5b566e4b1cabb2bc989833a43c" candidate_uuid: "3175c7c25e7244f396941e84ecf8410d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d41c1694404fcca967810240f07216"
I20260810 11:49:03.420915 7573 raft_consensus.cc:3037] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:03.420914 7282 raft_consensus.cc:3037] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:03.424278 7573 raft_consensus.cc:2454] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3175c7c25e7244f396941e84ecf8410d in term 1.
I20260810 11:49:03.424278 7282 raft_consensus.cc:2454] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3175c7c25e7244f396941e84ecf8410d in term 1.
I20260810 11:49:03.424731 7357 leader_election.cc:302] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3175c7c25e7244f396941e84ecf8410d, b8aa1690053f41579f1f3c8f96a46d7c; no voters:
I20260810 11:49:03.425086 7649 raft_consensus.cc:2781] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:03.425634 7649 raft_consensus.cc:686] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 1 LEADER]: Becoming Leader. State: Replica: 3175c7c25e7244f396941e84ecf8410d, State: Running, Role: LEADER
I20260810 11:49:03.426169 7649 consensus_queue.cc:237] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } }
I20260810 11:49:03.430855 7126 catalog_manager.cc:5495] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3175c7c25e7244f396941e84ecf8410d (127.20.93.130). New cstate: current_term: 1 leader_uuid: "3175c7c25e7244f396941e84ecf8410d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3175c7c25e7244f396941e84ecf8410d" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 38035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 } health_report { overall_health: UNKNOWN } } }
I20260810 11:49:03.453225 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.447642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.20.93.130:38035)
0810 11:49:03.447867 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.447874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.447895 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:49:03.448080 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.448085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.448178 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.448336 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:03.448735 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.449452 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.450964 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.451387 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.451534 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.452095 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:03.452615 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:03.452789 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.452849 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.452936 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:49:03.461875 7573 raft_consensus.cc:1261] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 1 FOLLOWER]: Refusing update from remote peer 3175c7c25e7244f396941e84ecf8410d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:03.461874 7282 raft_consensus.cc:1261] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 1 FOLLOWER]: Refusing update from remote peer 3175c7c25e7244f396941e84ecf8410d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:03.462949 7654 consensus_queue.cc:1035] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8aa1690053f41579f1f3c8f96a46d7c" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:03.463310 7649 consensus_queue.cc:1035] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d41c1694404fcca967810240f07216" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:03.491892 7126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:03.492233 7126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:03.496098 7126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1ec28328975470cb8c0b7b6aaf331f2: Sending DeleteTablet for 3 replicas of tablet a75d6d5b566e4b1cabb2bc989833a43c
I20260810 11:49:03.497439 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:03 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:36658
I20260810 11:49:03.497884 7551 tablet_service.cc:1514] Processing DeleteTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:03 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:51468
I20260810 11:49:03.498518 7406 tablet_service.cc:1514] Processing DeleteTablet for tablet a75d6d5b566e4b1cabb2bc989833a43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:03 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:50864
I20260810 11:49:03.499471 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7195
I20260810 11:49:03.513903 7670 tablet_replica.cc:330] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: stopping tablet replica
I20260810 11:49:03.514678 7671 tablet_replica.cc:330] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: stopping tablet replica
I20260810 11:49:03.515529 7671 raft_consensus.cc:2229] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:03.516173 7670 raft_consensus.cc:2229] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:03.516850 7670 raft_consensus.cc:2258] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:03.517632 7671 raft_consensus.cc:2258] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:03.520298 7670 ts_tablet_manager.cc:1905] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:03.523329 7669 tablet_replica.cc:330] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: stopping tablet replica
I20260810 11:49:03.523787 7669 raft_consensus.cc:2229] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:03.524331 7669 raft_consensus.cc:2258] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:03.524596 7671 ts_tablet_manager.cc:1905] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:49:03 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:49:03.529042 7669 ts_tablet_manager.cc:1905] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:03.536093 7671 ts_tablet_manager.cc:1918] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:03.536288 7671 log.cc:1196] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/a75d6d5b566e4b1cabb2bc989833a43c
I20260810 11:49:03.536378 7669 ts_tablet_manager.cc:1918] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:03.536559 7669 log.cc:1196] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/a75d6d5b566e4b1cabb2bc989833a43c
I20260810 11:49:03.536888 7671 ts_tablet_manager.cc:1939] T a75d6d5b566e4b1cabb2bc989833a43c P 3175c7c25e7244f396941e84ecf8410d: Deleting consensus metadata
I20260810 11:49:03.537108 7669 ts_tablet_manager.cc:1939] T a75d6d5b566e4b1cabb2bc989833a43c P 92d41c1694404fcca967810240f07216: Deleting consensus metadata
I20260810 11:49:03.538740 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.503127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.20.93.129:33591)
0810 11:49:03.503374 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.503383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.514941 (+ 11558us) server_negotiation.cc:407] Connection header received
0810 11:49:03.515253 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.515260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.515346 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.515426 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:03.517555 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.518847 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.520736 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.522039 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.522175 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.529797 (+ 7622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:03.529814 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:03.529833 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:03.529883 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:03.534189 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.535256 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.535266 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.535270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.535449 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.535979 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.535984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.535988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.536908 (+ 920us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:03.537368 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.538062 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.538331 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:49:03.538952 7112 catalog_manager.cc:4841] TS 92d41c1694404fcca967810240f07216 (127.20.93.129:33591): tablet a75d6d5b566e4b1cabb2bc989833a43c (table test-table [id=77dcf9487ac34e2f87afb50c75e55730]) successfully deleted
I20260810 11:49:03.539367 7113 catalog_manager.cc:4841] TS 3175c7c25e7244f396941e84ecf8410d (127.20.93.130:38035): tablet a75d6d5b566e4b1cabb2bc989833a43c (table test-table [id=77dcf9487ac34e2f87afb50c75e55730]) successfully deleted
I20260810 11:49:03.540663 7328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:03.546550 7670 ts_tablet_manager.cc:1918] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:03.546777 7670 log.cc:1196] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/a75d6d5b566e4b1cabb2bc989833a43c
I20260810 11:49:03.547267 7670 ts_tablet_manager.cc:1939] T a75d6d5b566e4b1cabb2bc989833a43c P b8aa1690053f41579f1f3c8f96a46d7c: Deleting consensus metadata
I20260810 11:49:03.548921 7113 catalog_manager.cc:4841] TS b8aa1690053f41579f1f3c8f96a46d7c (127.20.93.131:43485): tablet a75d6d5b566e4b1cabb2bc989833a43c (table test-table [id=77dcf9487ac34e2f87afb50c75e55730]) successfully deleted
I20260810 11:49:03.662943 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7195
I20260810 11:49:03.684787 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7339
Aug 10 11:49:03 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:49:03.703774 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.685759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.20.93.130:38035)
0810 11:49:03.685953 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.685960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.685982 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:49:03.686141 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.686147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.686286 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.686400 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:03.687750 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.689002 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.690535 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.691229 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.691361 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.696081 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:03.696102 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:03.696123 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:03.696178 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:03.700225 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.701280 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.701296 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.701300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.701465 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.701978 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.701984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.701987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.702506 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:03.702860 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.703127 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.703374 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:49:03.704540 7472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:03.821265 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7339
I20260810 11:49:03.842737 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7483
Aug 10 11:49:03 dist-test-slave-njxd krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:49:03.858511 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:03.843622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.20.93.131:43485)
0810 11:49:03.843838 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:03.843845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:03.843944 (+ 99us) server_negotiation.cc:407] Connection header received
0810 11:49:03.844159 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:03.844171 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:03.844240 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:03.844351 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:03.845759 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.846478 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.847864 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:03.848301 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:03.848439 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:03.853064 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:03.853080 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:03.853096 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:03.853144 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:03.855970 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.856622 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.856629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.856631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.856799 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:03.857148 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:03.857151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:03.857153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:03.857490 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:03.857808 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:03.858055 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 11:49:03.858239 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:49:03.859200 7616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:03.949976 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7483
I20260810 11:49:03.969794 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7094
I20260810 11:49:03.971076 7160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:04.087903 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7094
2026-08-10T11:49:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4799 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T11:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:04Z Disabled control of system clock
I20260810 11:49:04.182408 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35673
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:04.538370 7680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:04.538707 7680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:04.538795 7680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:04.546446 7680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:04.546583 7680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:04.546623 7680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:04.546675 7680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:04.546726 7680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:04.556313 7680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35673
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:04.557668 7680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:04.559581 7680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:04.571960 7686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:04.571913 7685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:04.571954 7688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:04.572901 7680 server_base.cc:1034] running on GCE node
I20260810 11:49:04.573786 7680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:04.575256 7680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:04.576465 7680 hybrid_clock.cc:648] HybridClock initialized: now 1786362544576430 us; error 33 us; skew 500 ppm
I20260810 11:49:04.576845 7680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:04.583794 7680 webserver.cc:460] Webserver started at http://127.20.93.190:45465/ using document root <none> and password file <none>
I20260810 11:49:04.584549 7680 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:04.584684 7680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:04.585048 7680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:04.588128 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "4b180a944f104704aa79d6749e3f8624"
format_stamp: "Formatted at 2026-08-10 11:49:04 on dist-test-slave-njxd"
server_key: "7ec35022b382e9ee29375b2d3c63f9a3"
server_key_iv: "293372eab3bf88deb6ede4ef934f0e43"
server_key_version: "encryptionkey@0"
I20260810 11:49:04.588979 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "4b180a944f104704aa79d6749e3f8624"
format_stamp: "Formatted at 2026-08-10 11:49:04 on dist-test-slave-njxd"
server_key: "7ec35022b382e9ee29375b2d3c63f9a3"
server_key_iv: "293372eab3bf88deb6ede4ef934f0e43"
server_key_version: "encryptionkey@0"
I20260810 11:49:04.594836 7680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 11:49:04.598521 7695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:04.600041 7680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:49:04.600248 7680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "4b180a944f104704aa79d6749e3f8624"
format_stamp: "Formatted at 2026-08-10 11:49:04 on dist-test-slave-njxd"
server_key: "7ec35022b382e9ee29375b2d3c63f9a3"
server_key_iv: "293372eab3bf88deb6ede4ef934f0e43"
server_key_version: "encryptionkey@0"
I20260810 11:49:04.600487 7680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:04.629621 7680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:04.630581 7680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:04.630895 7680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:04.650816 7680 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35673
I20260810 11:49:04.650836 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35673 every 8 connection(s)
I20260810 11:49:04.652339 7680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:04.653030 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7680
I20260810 11:49:04.653264 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:04.653927 20854 external_mini_cluster.cc:1428] Setting key 54e97a0899a8c3c4031d71071649d389
I20260810 11:49:04.656912 7751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:04.671831 7754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:04.656264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.20.93.190:35673)
0810 11:49:04.657609 (+ 1345us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:04.657629 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:04.657772 (+ 143us) server_negotiation.cc:407] Connection header received
0810 11:49:04.660157 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:04.660191 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:04.660892 (+ 701us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:04.661370 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:04.661982 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:04.664117 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:04.665805 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:04.666915 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:04.667127 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:04.668539 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:04.668556 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:04.668578 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:04.669435 (+ 857us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:04.669546 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:04.669618 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:04.669762 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:04.670034 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 11:49:04.670414 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":477,"threads_started":1}
I20260810 11:49:04.672427 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: Bootstrap starting.
I20260810 11:49:04.676231 7751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:04.677276 7751 log.cc:824] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:04.680102 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: No bootstrap required, opened a new log
I20260810 11:49:04.687740 7751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } }
I20260810 11:49:04.688102 7751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:04.688236 7751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b180a944f104704aa79d6749e3f8624, State: Initialized, Role: FOLLOWER
I20260810 11:49:04.688819 7751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } }
I20260810 11:49:04.689055 7751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:04.689219 7751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:04.689412 7751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:04.691704 7751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } }
I20260810 11:49:04.692258 7751 leader_election.cc:302] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b180a944f104704aa79d6749e3f8624; no voters:
I20260810 11:49:04.692989 7751 leader_election.cc:288] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:04.693186 7757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:04.694154 7757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [term 1 LEADER]: Becoming Leader. State: Replica: 4b180a944f104704aa79d6749e3f8624, State: Running, Role: LEADER
I20260810 11:49:04.694751 7757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } }
I20260810 11:49:04.695475 7751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:04.698117 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b180a944f104704aa79d6749e3f8624. Latest consensus state: current_term: 1 leader_uuid: "4b180a944f104704aa79d6749e3f8624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } } }
I20260810 11:49:04.698438 7758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b180a944f104704aa79d6749e3f8624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b180a944f104704aa79d6749e3f8624" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35673 } } }
I20260810 11:49:04.698707 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:04.698835 7758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:04.702018 7767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:04.707947 7767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:04.718201 7767 catalog_manager.cc:1347] Generated new cluster ID: 09d31c3bd953493bab3b037e4d6d779f
I20260810 11:49:04.718353 7767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:04.747408 7767 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:04.748973 7767 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:04.755987 7767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: Generated new TSK 0
I20260810 11:49:04.756912 7767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:04.766415 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--builtin_ntp_servers=127.20.93.148:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:05.150139 7778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:05.150470 7778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:05.150565 7778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:05.158071 7778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:05.158205 7778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:05.158325 7778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:05.167770 7778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:05.169162 7778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:05.171068 7778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:05.185662 7783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:05.185727 7784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:05.185722 7786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:05.186667 7778 server_base.cc:1034] running on GCE node
I20260810 11:49:05.187412 7778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:05.188650 7778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:05.189924 7778 hybrid_clock.cc:648] HybridClock initialized: now 1786362545189856 us; error 60 us; skew 500 ppm
I20260810 11:49:05.190328 7778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:05.193058 7778 webserver.cc:460] Webserver started at http://127.20.93.129:35399/ using document root <none> and password file <none>
I20260810 11:49:05.193939 7778 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:05.194103 7778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:05.194412 7778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:05.198096 7778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "04d35de46ca3406d9b23fe11e31c8ec3"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "06ad33869f00d8ca504e07f907a44e10"
server_key_iv: "986a205a4146b7a45d59ece6f534e8f5"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.198944 7778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "04d35de46ca3406d9b23fe11e31c8ec3"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "06ad33869f00d8ca504e07f907a44e10"
server_key_iv: "986a205a4146b7a45d59ece6f534e8f5"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.205215 7778 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:49:05.209260 7795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.210940 7778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:05.211124 7778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "04d35de46ca3406d9b23fe11e31c8ec3"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "06ad33869f00d8ca504e07f907a44e10"
server_key_iv: "986a205a4146b7a45d59ece6f534e8f5"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.211330 7778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:05.230263 7778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:05.231145 7778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:05.231432 7778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:05.232513 7778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:05.234158 7778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:05.234249 7778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.234326 7778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:05.234387 7778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.269789 7778 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:44019
I20260810 11:49:05.269850 7914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:44019 every 8 connection(s)
I20260810 11:49:05.271282 7778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:05.276077 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7778
I20260810 11:49:05.276285 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:05.276932 20854 external_mini_cluster.cc:1428] Setting key 2c8719acb52af2e07a642dd32d8e643a
I20260810 11:49:05.280476 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--builtin_ntp_servers=127.20.93.148:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:05.287315 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:05.274127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:49321 (local address 127.20.93.190:35673)
0810 11:49:05.274632 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:05.274637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:05.276030 (+ 1393us) server_negotiation.cc:407] Connection header received
0810 11:49:05.278569 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:05.278576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:05.278654 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:05.278783 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:05.280273 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.281134 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.283869 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.284316 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.284437 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:05.286429 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:05.286435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:05.286438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:05.286460 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:05.286505 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:05.286519 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:05.286608 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:05.286930 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 11:49:05.287027 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":263,"threads_started":1}
I20260810 11:49:05.288882 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:05.274816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35673 (local address 127.20.93.129:49321)
0810 11:49:05.275821 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:05.275875 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:05.278265 (+ 2390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:05.279061 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:05.279074 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:05.279206 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:05.280059 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:05.280089 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.281314 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.281323 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:05.283658 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:05.283680 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.284588 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.284599 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:05.284966 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:05.285976 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:05.285982 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:05.286045 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:05.286182 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:05.286630 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:05.286645 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:05.286656 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:49:05.286859 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 11:49:05.287366 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":271,"threads_started":1}
I20260810 11:49:05.290709 7915 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35673
I20260810 11:49:05.291072 7915 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:05.291877 7915 heartbeater.cc:511] Master 127.20.93.190:35673 requested a full tablet report, sending...
I20260810 11:49:05.294140 7712 ts_manager.cc:194] Registered new tserver with Master: 04d35de46ca3406d9b23fe11e31c8ec3 (127.20.93.129:44019)
I20260810 11:49:05.295840 7712 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.129:49321
W20260810 11:49:05.637073 7921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:05.637387 7921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:05.637542 7921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:05.645555 7921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:05.645740 7921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:05.645920 7921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:05.655489 7921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:05.656870 7921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:05.658775 7921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:05.672318 7926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:05.672286 7930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:05.672240 7927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:05.673039 7921 server_base.cc:1034] running on GCE node
I20260810 11:49:05.673760 7921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:05.674731 7921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:05.675945 7921 hybrid_clock.cc:648] HybridClock initialized: now 1786362545675892 us; error 42 us; skew 500 ppm
I20260810 11:49:05.676294 7921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:05.678643 7921 webserver.cc:460] Webserver started at http://127.20.93.130:41181/ using document root <none> and password file <none>
I20260810 11:49:05.679384 7921 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:05.679507 7921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:05.679849 7921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:05.682930 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "dd24fef297f74aadb0628f33541315dc"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "b647fd16eddcf54037a63d96ce12309f"
server_key_iv: "59a761383fbd1856ad755fbf4373a767"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.683816 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "dd24fef297f74aadb0628f33541315dc"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "b647fd16eddcf54037a63d96ce12309f"
server_key_iv: "59a761383fbd1856ad755fbf4373a767"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.689272 7921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:49:05.692860 7936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.694340 7921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:49:05.694538 7921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "dd24fef297f74aadb0628f33541315dc"
format_stamp: "Formatted at 2026-08-10 11:49:05 on dist-test-slave-njxd"
server_key: "b647fd16eddcf54037a63d96ce12309f"
server_key_iv: "59a761383fbd1856ad755fbf4373a767"
server_key_version: "encryptionkey@0"
I20260810 11:49:05.694772 7921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:05.716974 7921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:05.717954 7921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:05.718284 7921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:05.719327 7921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:05.720924 7921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:05.721084 7921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.721230 7921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:05.721318 7921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:05.755673 7921 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:41465
I20260810 11:49:05.755776 8055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:41465 every 8 connection(s)
I20260810 11:49:05.757048 7921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:05.757432 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 7921
I20260810 11:49:05.757680 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:05.758327 20854 external_mini_cluster.cc:1428] Setting key 9c6dd73cc7f6df6a1d8c17bce4381ab5
I20260810 11:49:05.761420 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--builtin_ntp_servers=127.20.93.148:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:05.771359 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:05.759786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:51501 (local address 127.20.93.190:35673)
0810 11:49:05.760008 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:05.760015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:05.761737 (+ 1722us) server_negotiation.cc:407] Connection header received
0810 11:49:05.763625 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:05.763633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:05.763730 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:05.763847 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:05.764929 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.766109 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.767889 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.768693 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.768852 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:05.770377 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:05.770386 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:05.770394 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:05.770422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:05.770466 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:05.770484 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:05.770585 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:05.770811 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 11:49:05.770932 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:49:05.772231 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:05.760440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35673 (local address 127.20.93.130:51501)
0810 11:49:05.761547 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:05.761617 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:05.763339 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:05.764065 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:05.764074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:05.764164 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:05.764751 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:05.764768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.766284 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.766288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:05.767728 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:05.767741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:05.768959 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:05.768970 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:05.769196 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:05.770021 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:05.770024 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:05.770074 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:05.770163 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:05.770589 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:05.770597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:05.770603 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:49:05.770765 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 11:49:05.771081 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":347,"threads_started":1}
I20260810 11:49:05.773866 8056 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35673
I20260810 11:49:05.774240 8056 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:05.775074 8056 heartbeater.cc:511] Master 127.20.93.190:35673 requested a full tablet report, sending...
I20260810 11:49:05.776813 7712 ts_manager.cc:194] Registered new tserver with Master: dd24fef297f74aadb0628f33541315dc (127.20.93.130:41465)
I20260810 11:49:05.778110 7712 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.130:51501
W20260810 11:49:06.110816 8059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:06.111109 8059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:06.111195 8059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:06.118693 8059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:06.118801 8059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:06.118911 8059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:06.128118 8059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:35673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:06.129354 8059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:06.131161 8059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:06.144100 8064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:06.144151 8067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:06.144170 8065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:06.145139 8059 server_base.cc:1034] running on GCE node
I20260810 11:49:06.145767 8059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:06.146746 8059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:06.147996 8059 hybrid_clock.cc:648] HybridClock initialized: now 1786362546147958 us; error 45 us; skew 500 ppm
I20260810 11:49:06.148316 8059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:06.150662 8059 webserver.cc:460] Webserver started at http://127.20.93.131:38391/ using document root <none> and password file <none>
I20260810 11:49:06.151403 8059 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:06.151547 8059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:06.151876 8059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:06.154907 8059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "edea6e2146be461d8ede12ac7f45026d"
format_stamp: "Formatted at 2026-08-10 11:49:06 on dist-test-slave-njxd"
server_key: "0670ad1a4925a5d46ec074a87feba840"
server_key_iv: "79c64cefc810f67fa81c17b4495cc18e"
server_key_version: "encryptionkey@0"
I20260810 11:49:06.155758 8059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "edea6e2146be461d8ede12ac7f45026d"
format_stamp: "Formatted at 2026-08-10 11:49:06 on dist-test-slave-njxd"
server_key: "0670ad1a4925a5d46ec074a87feba840"
server_key_iv: "79c64cefc810f67fa81c17b4495cc18e"
server_key_version: "encryptionkey@0"
I20260810 11:49:06.161015 8059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:06.164651 8075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:06.166069 8059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:06.166246 8059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "edea6e2146be461d8ede12ac7f45026d"
format_stamp: "Formatted at 2026-08-10 11:49:06 on dist-test-slave-njxd"
server_key: "0670ad1a4925a5d46ec074a87feba840"
server_key_iv: "79c64cefc810f67fa81c17b4495cc18e"
server_key_version: "encryptionkey@0"
I20260810 11:49:06.166486 8059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:06.178404 8059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:06.179239 8059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:06.179564 8059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:06.180581 8059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:06.182216 8059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:06.182343 8059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:06.182483 8059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:06.182580 8059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:06.215998 8059 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:40979
I20260810 11:49:06.216033 8195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:40979 every 8 connection(s)
I20260810 11:49:06.217446 8059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:06.220153 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8059
I20260810 11:49:06.220335 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:06.220830 20854 external_mini_cluster.cc:1428] Setting key 2c5a8730630f8ffe44ea5e8255c1826a
I20260810 11:49:06.230518 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.220141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:46599 (local address 127.20.93.190:35673)
0810 11:49:06.220404 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.220411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.222110 (+ 1699us) server_negotiation.cc:407] Connection header received
0810 11:49:06.224402 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.224406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.224470 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.224550 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:06.225623 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.226513 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.228059 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.228457 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.228552 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.229845 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:06.229849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:06.229852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:06.229870 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:06.229901 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:06.229913 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:06.229983 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.230203 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.230277 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:49:06.231602 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.220722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:35673 (local address 127.20.93.131:46599)
0810 11:49:06.221942 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.221987 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.224158 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.224753 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.224761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.224854 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:06.225439 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.225457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.226654 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.226658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.227929 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.227941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.228643 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.228648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.228847 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.229486 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:06.229489 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:06.229536 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:06.229674 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:06.229996 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:06.230004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:06.230010 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.230172 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.230504 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":321,"threads_started":1}
I20260810 11:49:06.232945 8197 heartbeater.cc:347] Connected to a master server at 127.20.93.190:35673
I20260810 11:49:06.233259 8197 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:06.234047 8197 heartbeater.cc:511] Master 127.20.93.190:35673 requested a full tablet report, sending...
I20260810 11:49:06.235641 7712 ts_manager.cc:194] Registered new tserver with Master: edea6e2146be461d8ede12ac7f45026d (127.20.93.131:40979)
I20260810 11:49:06.236383 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:06.236528 7712 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.131:46599
I20260810 11:49:06.247210 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.241106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.20.93.190:35673)
0810 11:49:06.241336 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.241344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.241751 (+ 407us) server_negotiation.cc:407] Connection header received
0810 11:49:06.241949 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.241954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.242014 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.242116 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:06.242606 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.243274 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.244750 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.245157 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.245299 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.246485 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:06.246490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:06.246493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:06.246511 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:06.246541 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:06.246553 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:06.246649 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.246802 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.246894 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:49:06.251010 7712 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:06.257977 7712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:06.295972 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.273212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44019 (local address 127.0.0.1:54732)
0810 11:49:06.274385 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.274409 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.274598 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.285532 (+ 10934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.285542 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.285635 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:06.286234 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.286253 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.290208 (+ 3955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.290214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.292792 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.292807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.294842 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.294849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.295096 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.295151 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.295301 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.295568 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":49,"thread_start_us":872,"threads_started":1}
I20260810 11:49:06.296012 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.273880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.20.93.129:44019)
0810 11:49:06.284333 (+ 10453us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.284357 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.284401 (+ 44us) server_negotiation.cc:407] Connection header received
0810 11:49:06.284530 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.284540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.284894 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.286542 (+ 1648us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:06.286846 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.288984 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.293023 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.294616 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.294761 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.295051 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.295364 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.295580 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10203,"thread_start_us":11240,"threads_started":1}
I20260810 11:49:06.298239 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.271403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:41465 (local address 127.0.0.1:42552)
0810 11:49:06.272202 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.272250 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.272423 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.290837 (+ 18414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.290842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.290872 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:06.291288 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.291301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.293160 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.293166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.295394 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.295408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.297405 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.297411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.297613 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.297647 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.297733 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.297863 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":29,"thread_start_us":351,"threads_started":1}
I20260810 11:49:06.298285 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.271693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.20.93.130:41465)
0810 11:49:06.290127 (+ 18434us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.290139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.290180 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:49:06.290305 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.290317 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.290610 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.290825 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:06.291498 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.292994 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.295643 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.297223 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.297371 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.297629 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.297783 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.297943 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18201,"thread_start_us":18053,"threads_started":1}
I20260810 11:49:06.298653 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.277276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:40979 (local address 127.0.0.1:58394)
0810 11:49:06.290700 (+ 13424us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.290723 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.290850 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.291441 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.291446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.291471 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:06.291874 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.291884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.293604 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.293610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.295655 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.295669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.297866 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.297873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.298056 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.298087 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.298185 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.298308 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12207,"mutex_wait_us":59,"thread_start_us":12028,"threads_started":1}
I20260810 11:49:06.298951 7915 heartbeater.cc:503] Master 127.20.93.190:35673 was elected leader, sending a full tablet report...
I20260810 11:49:06.299410 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.278094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.20.93.131:40979)
0810 11:49:06.290682 (+ 12588us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.290690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.290996 (+ 306us) server_negotiation.cc:407] Connection header received
0810 11:49:06.291100 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.291104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.291263 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.291456 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:06.292075 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.293375 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.295873 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.297302 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.297424 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.297726 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.298894 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.299064 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12414,"thread_start_us":12239,"threads_started":1}
I20260810 11:49:06.300865 7847 tablet_service.cc:1467] Processing CreateTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (DEFAULT_TABLE table=test-table [id=454f3c4379b44dbdb2976f32ac2151cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:06.303002 7847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:06.305270 7986 tablet_service.cc:1467] Processing CreateTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (DEFAULT_TABLE table=test-table [id=454f3c4379b44dbdb2976f32ac2151cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:06.305436 8126 tablet_service.cc:1467] Processing CreateTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (DEFAULT_TABLE table=test-table [id=454f3c4379b44dbdb2976f32ac2151cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:06.306466 7986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:06.306903 8126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:06.317370 8215 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Bootstrap starting.
I20260810 11:49:06.319382 8216 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Bootstrap starting.
I20260810 11:49:06.321877 8215 tablet_bootstrap.cc:654] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:06.321965 8217 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Bootstrap starting.
I20260810 11:49:06.322993 8216 tablet_bootstrap.cc:654] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:06.323323 8215 log.cc:824] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:06.324244 8216 log.cc:824] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:06.326263 8217 tablet_bootstrap.cc:654] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:06.326906 8215 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: No bootstrap required, opened a new log
I20260810 11:49:06.327194 8215 ts_tablet_manager.cc:1397] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 11:49:06.327508 8217 log.cc:824] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:06.328154 8216 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: No bootstrap required, opened a new log
I20260810 11:49:06.328464 8216 ts_tablet_manager.cc:1397] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:49:06.330714 8217 tablet_bootstrap.cc:492] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: No bootstrap required, opened a new log
I20260810 11:49:06.330967 8217 ts_tablet_manager.cc:1397] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 11:49:06.336606 8216 raft_consensus.cc:348] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.337001 8216 raft_consensus.cc:374] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:06.337098 8216 raft_consensus.cc:729] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d35de46ca3406d9b23fe11e31c8ec3, State: Initialized, Role: FOLLOWER
I20260810 11:49:06.337873 8216 consensus_queue.cc:260] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.337932 8215 raft_consensus.cc:348] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.338361 8215 raft_consensus.cc:374] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:06.338455 8215 raft_consensus.cc:729] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd24fef297f74aadb0628f33541315dc, State: Initialized, Role: FOLLOWER
I20260810 11:49:06.339133 8215 consensus_queue.cc:260] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.340512 8056 heartbeater.cc:503] Master 127.20.93.190:35673 was elected leader, sending a full tablet report...
I20260810 11:49:06.341053 8215 ts_tablet_manager.cc:1428] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260810 11:49:06.341760 8217 raft_consensus.cc:348] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.342249 8217 raft_consensus.cc:374] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:06.342342 8217 raft_consensus.cc:729] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edea6e2146be461d8ede12ac7f45026d, State: Initialized, Role: FOLLOWER
I20260810 11:49:06.342747 8216 ts_tablet_manager.cc:1428] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260810 11:49:06.342994 8217 consensus_queue.cc:260] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.344274 8197 heartbeater.cc:503] Master 127.20.93.190:35673 was elected leader, sending a full tablet report...
I20260810 11:49:06.344944 8217 ts_tablet_manager.cc:1428] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 11:49:06.411967 8224 raft_consensus.cc:482] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:06.412245 8224 raft_consensus.cc:504] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.413973 8224 leader_election.cc:288] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd24fef297f74aadb0628f33541315dc (127.20.93.130:41465), 04d35de46ca3406d9b23fe11e31c8ec3 (127.20.93.129:44019)
I20260810 11:49:06.420197 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.414354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:39723 (local address 127.20.93.130:41465)
0810 11:49:06.414575 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.414588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.414981 (+ 393us) server_negotiation.cc:407] Connection header received
0810 11:49:06.415166 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.415170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.415246 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.415354 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:06.415819 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.416712 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.418544 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.419309 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.419471 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.419546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.419802 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.419926 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:49:06.420248 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.414448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:41465 (local address 127.20.93.131:39723)
0810 11:49:06.414688 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.414703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.414946 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.415426 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.415431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.415464 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:06.415637 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.415646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.416847 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.416854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.418370 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.418382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.419472 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.419478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.419646 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.419670 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.419774 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.419914 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":16}
I20260810 11:49:06.420861 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.414703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:55565 (local address 127.20.93.129:44019)
0810 11:49:06.414931 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.414938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.415689 (+ 751us) server_negotiation.cc:407] Connection header received
0810 11:49:06.415915 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.415918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.415986 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.416103 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:06.416612 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.417295 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.419111 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.419907 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.420075 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.420142 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.420475 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.420602 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:49:06.421308 8008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ba6c5f8fa54a23a9cd7deb767e2f16" candidate_uuid: "edea6e2146be461d8ede12ac7f45026d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd24fef297f74aadb0628f33541315dc" is_pre_election: true
I20260810 11:49:06.421557 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.414485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:44019 (local address 127.20.93.131:55565)
0810 11:49:06.415575 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:06.415591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:06.415760 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:06.416133 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:06.416137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:06.416160 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:06.416463 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.416470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.417447 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.417453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.418921 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:06.418934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.420087 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.420093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:06.420252 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.420306 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 11:49:06.421057 (+ 751us) client_negotiation.cc:240] Negotiation successful
0810 11:49:06.421210 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":17,"thread_start_us":769,"threads_started":1}
I20260810 11:49:06.422006 8008 raft_consensus.cc:2454] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edea6e2146be461d8ede12ac7f45026d in term 0.
I20260810 11:49:06.422811 8079 leader_election.cc:302] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd24fef297f74aadb0628f33541315dc, edea6e2146be461d8ede12ac7f45026d; no voters:
I20260810 11:49:06.422736 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ba6c5f8fa54a23a9cd7deb767e2f16" candidate_uuid: "edea6e2146be461d8ede12ac7f45026d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" is_pre_election: true
I20260810 11:49:06.423224 8224 raft_consensus.cc:2781] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:06.423234 7868 raft_consensus.cc:2454] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edea6e2146be461d8ede12ac7f45026d in term 0.
I20260810 11:49:06.423348 8224 raft_consensus.cc:482] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:06.423471 8224 raft_consensus.cc:3037] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:06.425943 8224 raft_consensus.cc:504] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.426604 8224 leader_election.cc:288] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [CANDIDATE]: Term 1 election: Requested vote from peers dd24fef297f74aadb0628f33541315dc (127.20.93.130:41465), 04d35de46ca3406d9b23fe11e31c8ec3 (127.20.93.129:44019)
I20260810 11:49:06.427112 8008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ba6c5f8fa54a23a9cd7deb767e2f16" candidate_uuid: "edea6e2146be461d8ede12ac7f45026d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd24fef297f74aadb0628f33541315dc"
I20260810 11:49:06.427204 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ba6c5f8fa54a23a9cd7deb767e2f16" candidate_uuid: "edea6e2146be461d8ede12ac7f45026d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d35de46ca3406d9b23fe11e31c8ec3"
I20260810 11:49:06.427357 8008 raft_consensus.cc:3037] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:06.427416 7868 raft_consensus.cc:3037] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:06.430804 7868 raft_consensus.cc:2454] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edea6e2146be461d8ede12ac7f45026d in term 1.
I20260810 11:49:06.430804 8008 raft_consensus.cc:2454] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edea6e2146be461d8ede12ac7f45026d in term 1.
I20260810 11:49:06.431281 8079 leader_election.cc:302] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd24fef297f74aadb0628f33541315dc, edea6e2146be461d8ede12ac7f45026d; no voters:
I20260810 11:49:06.431640 8224 raft_consensus.cc:2781] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:06.432092 8224 raft_consensus.cc:686] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 1 LEADER]: Becoming Leader. State: Replica: edea6e2146be461d8ede12ac7f45026d, State: Running, Role: LEADER
I20260810 11:49:06.432587 8224 consensus_queue.cc:237] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } }
I20260810 11:49:06.438063 7711 catalog_manager.cc:5495] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d reported cstate change: term changed from 0 to 1, leader changed from <none> to edea6e2146be461d8ede12ac7f45026d (127.20.93.131). New cstate: current_term: 1 leader_uuid: "edea6e2146be461d8ede12ac7f45026d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edea6e2146be461d8ede12ac7f45026d" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 40979 } health_report { overall_health: HEALTHY } } }
W20260810 11:49:06.469110 8198 tablet.cc:2369] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:06.470139 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.464648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.20.93.131:40979)
0810 11:49:06.464831 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.464835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.464855 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:49:06.464944 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.464948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.465044 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.465178 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:06.465543 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.466384 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.467812 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.468225 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.468332 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.468756 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:06.469343 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:06.469485 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.469733 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.469887 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:49:06.478523 7868 raft_consensus.cc:1261] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 1 FOLLOWER]: Refusing update from remote peer edea6e2146be461d8ede12ac7f45026d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:06.478523 8008 raft_consensus.cc:1261] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 1 FOLLOWER]: Refusing update from remote peer edea6e2146be461d8ede12ac7f45026d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:06.479506 8226 consensus_queue.cc:1035] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd24fef297f74aadb0628f33541315dc" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:06.479895 8224 consensus_queue.cc:1035] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d35de46ca3406d9b23fe11e31c8ec3" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 44019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:06.490587 8229 mvcc.cc:205] Tried to move back new op lower bound from 7316940990366003200 to 7316940990192578560. Current Snapshot: MvccSnapshot[applied={T|T < 7316940990366003200}]
I20260810 11:49:06.499046 8230 mvcc.cc:205] Tried to move back new op lower bound from 7316940990366003200 to 7316940990192578560. Current Snapshot: MvccSnapshot[applied={T|T < 7316940990366003200 or (T in {7316940990366003200})}]
I20260810 11:49:06.508062 7712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:06.508415 7712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:06.512578 7712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b180a944f104704aa79d6749e3f8624: Sending DeleteTablet for 3 replicas of tablet 83ba6c5f8fa54a23a9cd7deb767e2f16
I20260810 11:49:06.513964 8126 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:06 UTC) from {username='slave'} at 127.0.0.1:58394
I20260810 11:49:06.513932 7986 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:06 UTC) from {username='slave'} at 127.0.0.1:42552
I20260810 11:49:06.514793 7847 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:06 UTC) from {username='slave'} at 127.0.0.1:54732
I20260810 11:49:06.515027 8239 tablet_replica.cc:330] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: stopping tablet replica
I20260810 11:49:06.515498 8239 raft_consensus.cc:2229] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:06.516038 8239 raft_consensus.cc:2258] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:06.516786 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7778
I20260810 11:49:06.517987 8240 tablet_replica.cc:330] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: stopping tablet replica
I20260810 11:49:06.518397 8240 raft_consensus.cc:2229] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:06.519194 8240 raft_consensus.cc:2258] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:06.521335 8239 ts_tablet_manager.cc:1905] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:06.525403 8241 tablet_replica.cc:330] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: stopping tablet replica
I20260810 11:49:06.526036 8241 raft_consensus.cc:2229] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:06.526751 8241 raft_consensus.cc:2258] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:06.528609 8239 ts_tablet_manager.cc:1918] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:06.528779 8239 log.cc:1196] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/83ba6c5f8fa54a23a9cd7deb767e2f16
I20260810 11:49:06.529178 8241 ts_tablet_manager.cc:1905] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:06.529384 8239 ts_tablet_manager.cc:1939] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P dd24fef297f74aadb0628f33541315dc: Deleting consensus metadata
I20260810 11:49:06.531136 8240 ts_tablet_manager.cc:1905] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:06.536455 8241 ts_tablet_manager.cc:1918] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:06.536621 8241 log.cc:1196] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/83ba6c5f8fa54a23a9cd7deb767e2f16
I20260810 11:49:06.537206 8241 ts_tablet_manager.cc:1939] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P 04d35de46ca3406d9b23fe11e31c8ec3: Deleting consensus metadata
I20260810 11:49:06.538290 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.517928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.20.93.129:44019)
0810 11:49:06.518137 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.518146 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.527915 (+ 9769us) server_negotiation.cc:407] Connection header received
0810 11:49:06.528235 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.528243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.528354 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.528437 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:06.529398 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.530658 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.532786 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.533463 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.533649 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.536265 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:06.536277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:06.536297 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:06.537142 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:06.537214 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:06.537241 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:06.537336 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.537778 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.537904 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:49:06.539181 8240 ts_tablet_manager.cc:1918] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:06.539227 7696 catalog_manager.cc:4841] TS 04d35de46ca3406d9b23fe11e31c8ec3 (127.20.93.129:44019): tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (table test-table [id=454f3c4379b44dbdb2976f32ac2151cc]) successfully deleted
I20260810 11:49:06.539374 8240 log.cc:1196] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/83ba6c5f8fa54a23a9cd7deb767e2f16
I20260810 11:49:06.539939 8240 ts_tablet_manager.cc:1939] T 83ba6c5f8fa54a23a9cd7deb767e2f16 P edea6e2146be461d8ede12ac7f45026d: Deleting consensus metadata
I20260810 11:49:06.540375 7910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:06.540443 7698 catalog_manager.cc:4841] TS dd24fef297f74aadb0628f33541315dc (127.20.93.130:41465): tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (table test-table [id=454f3c4379b44dbdb2976f32ac2151cc]) successfully deleted
I20260810 11:49:06.541623 7699 catalog_manager.cc:4841] TS edea6e2146be461d8ede12ac7f45026d (127.20.93.131:40979): tablet 83ba6c5f8fa54a23a9cd7deb767e2f16 (table test-table [id=454f3c4379b44dbdb2976f32ac2151cc]) successfully deleted
I20260810 11:49:06.632663 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7778
I20260810 11:49:06.655840 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7921
I20260810 11:49:06.664971 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.656781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.20.93.130:41465)
0810 11:49:06.656983 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.656991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.657014 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:49:06.657087 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.657094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.657225 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.657345 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:06.657851 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.659037 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.660959 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.661722 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.661857 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.663191 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:06.663205 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:06.663225 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:06.664065 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:06.664137 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:06.664164 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:06.664305 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.664463 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.664590 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:49:06.665815 8051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:06.765921 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7921
I20260810 11:49:06.793177 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8059
I20260810 11:49:06.801769 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:06.793995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.20.93.131:40979)
0810 11:49:06.794184 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:06.794189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:06.794287 (+ 98us) server_negotiation.cc:407] Connection header received
0810 11:49:06.794484 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:06.794488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:06.794577 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:06.794666 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:06.795237 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.796034 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.798173 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:06.798731 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:06.798839 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:06.800165 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:06.800176 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:06.800191 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:06.800821 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:06.800877 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:06.800897 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:06.800983 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:06.801189 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 11:49:06.801329 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:49:06.802597 8190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:06.925192 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8059
I20260810 11:49:06.948055 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 7680
I20260810 11:49:06.949327 7746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:07.064706 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 7680
2026-08-10T11:49:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2977 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:07 dist-test-slave-njxd krb5kdc[8250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:07 dist-test-slave-njxd krb5kdc[8250](info): set up 2 sockets
Aug 10 11:49:07 dist-test-slave-njxd krb5kdc[8250](info): commencing operation
krb5kdc: starting...
W20260810 11:49:08.253728 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:08 dist-test-slave-njxd krb5kdc[8250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:08.582677 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42881
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:42881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:08.942003 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:08.942310 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:08.942534 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:08.950067 8266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:08.950203 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:08.950291 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:08.950381 8266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:08.950448 8266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:08.960922 8266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:42881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:08.962481 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:08.963923 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:08.976490 8274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:08.976526 8272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:08.976451 8271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:08.977514 8266 server_base.cc:1034] running on GCE node
I20260810 11:49:08.978399 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:08.979887 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:08.981122 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786362548981049 us; error 67 us; skew 500 ppm
I20260810 11:49:08.981539 8266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:08 dist-test-slave-njxd krb5kdc[8250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:08.984885 8266 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:08.986595 8266 webserver.cc:460] Webserver started at http://127.20.93.190:45973/ using document root <none> and password file <none>
I20260810 11:49:08.987287 8266 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:08.987394 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:08.987669 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:08.990867 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "4fed4d05c54b4e1db1bf98adec36ce97"
format_stamp: "Formatted at 2026-08-10 11:49:08 on dist-test-slave-njxd"
server_key: "8b3aebe7612384aa60bdbadc68bf4b0d"
server_key_iv: "bce276d3eb592d1dbdf472b4facbe684"
server_key_version: "encryptionkey@0"
I20260810 11:49:08.991604 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "4fed4d05c54b4e1db1bf98adec36ce97"
format_stamp: "Formatted at 2026-08-10 11:49:08 on dist-test-slave-njxd"
server_key: "8b3aebe7612384aa60bdbadc68bf4b0d"
server_key_iv: "bce276d3eb592d1dbdf472b4facbe684"
server_key_version: "encryptionkey@0"
I20260810 11:49:08.997272 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:09.000944 8281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:09.002533 8266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:09.002740 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "4fed4d05c54b4e1db1bf98adec36ce97"
format_stamp: "Formatted at 2026-08-10 11:49:08 on dist-test-slave-njxd"
server_key: "8b3aebe7612384aa60bdbadc68bf4b0d"
server_key_iv: "bce276d3eb592d1dbdf472b4facbe684"
server_key_version: "encryptionkey@0"
I20260810 11:49:09.002995 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:09.032321 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:09.037166 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:09.037492 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:09.056516 8266 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:42881
I20260810 11:49:09.056540 8337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:42881 every 8 connection(s)
I20260810 11:49:09.058109 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:09.062134 8338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:09.065714 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8266
I20260810 11:49:09.065905 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:09.066493 20854 external_mini_cluster.cc:1428] Setting key a110c1cd4b09ae804a9790f642956127
I20260810 11:49:09.078294 8338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: Bootstrap starting.
I20260810 11:49:09.082319 8338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:09.083495 8338 log.cc:824] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:49:09 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:09.086772 8338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: No bootstrap required, opened a new log
I20260810 11:49:09.094520 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:09.068658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.20.93.190:42881)
0810 11:49:09.069611 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:09.069632 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:09.069722 (+ 90us) server_negotiation.cc:407] Connection header received
0810 11:49:09.072719 (+ 2997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:09.072755 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:09.073415 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:09.073956 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:09.076412 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.078881 (+ 2469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.080694 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.081450 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.081715 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:09.086304 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:09.086312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:09.086330 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:09.086376 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:09.089833 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:09.090693 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:09.090705 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:09.090716 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:09.090913 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:09.091330 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:09.091342 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:09.091350 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:09.091978 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:09.092340 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:09.092837 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 11:49:09.093349 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":347,"threads_started":1}
I20260810 11:49:09.095319 8338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } }
I20260810 11:49:09.095803 8338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:09.095920 8338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fed4d05c54b4e1db1bf98adec36ce97, State: Initialized, Role: FOLLOWER
I20260810 11:49:09.096572 8338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } }
I20260810 11:49:09.096836 8338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:09.096989 8338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:09.097188 8338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:09.099656 8338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } }
I20260810 11:49:09.100175 8338 leader_election.cc:302] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fed4d05c54b4e1db1bf98adec36ce97; no voters:
I20260810 11:49:09.100754 8338 leader_election.cc:288] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:09.100916 8344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:09.102288 8344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [term 1 LEADER]: Becoming Leader. State: Replica: 4fed4d05c54b4e1db1bf98adec36ce97, State: Running, Role: LEADER
I20260810 11:49:09.102864 8344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } }
I20260810 11:49:09.103112 8338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:09.106465 8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fed4d05c54b4e1db1bf98adec36ce97. Latest consensus state: current_term: 1 leader_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } } }
I20260810 11:49:09.107213 8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:09.106542 8345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fed4d05c54b4e1db1bf98adec36ce97" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 42881 } } }
I20260810 11:49:09.107971 8345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:09.110327 8353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:09.115140 8353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:09.124449 8353 catalog_manager.cc:1347] Generated new cluster ID: b2303bdbcf48444bb9de19f0593b97a4
I20260810 11:49:09.124577 8353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:09.139250 8353 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:09.141155 8353 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:09.153143 8353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: Generated new TSK 0
I20260810 11:49:09.154091 8353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:49:09.237793 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:09.596423 8366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:09.596794 8366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:09.596971 8366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:09.604954 8366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:09.605113 8366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:09.605316 8366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:09.616129 8366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:09.617537 8366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:09.619094 8366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:09.633986 8371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:09.634068 8372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:09.634099 8374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:09.634851 8366 server_base.cc:1034] running on GCE node
I20260810 11:49:09.635507 8366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:09.636515 8366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:09.637763 8366 hybrid_clock.cc:648] HybridClock initialized: now 1786362549637725 us; error 33 us; skew 500 ppm
I20260810 11:49:09.638172 8366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:09 dist-test-slave-njxd krb5kdc[8250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362549, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:09.644680 8366 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:49:09.646384 8366 webserver.cc:460] Webserver started at http://127.20.93.129:36273/ using document root <none> and password file <none>
I20260810 11:49:09.647114 8366 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:09.647229 8366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:09.647521 8366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:09.650712 8366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "914c7ec8d90446beb82764e47381af3a"
format_stamp: "Formatted at 2026-08-10 11:49:09 on dist-test-slave-njxd"
server_key: "efd4c3bec170f0b70e9db00abc11f7f5"
server_key_iv: "f510c6868a2d2abd0a6e746ff67a05af"
server_key_version: "encryptionkey@0"
I20260810 11:49:09.651468 8366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "914c7ec8d90446beb82764e47381af3a"
format_stamp: "Formatted at 2026-08-10 11:49:09 on dist-test-slave-njxd"
server_key: "efd4c3bec170f0b70e9db00abc11f7f5"
server_key_iv: "f510c6868a2d2abd0a6e746ff67a05af"
server_key_version: "encryptionkey@0"
I20260810 11:49:09.657567 8366 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:49:09.661386 8381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:09.662947 8366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:09.663141 8366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "914c7ec8d90446beb82764e47381af3a"
format_stamp: "Formatted at 2026-08-10 11:49:09 on dist-test-slave-njxd"
server_key: "efd4c3bec170f0b70e9db00abc11f7f5"
server_key_iv: "f510c6868a2d2abd0a6e746ff67a05af"
server_key_version: "encryptionkey@0"
I20260810 11:49:09.663357 8366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:09.699891 8366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:09.704854 8366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:09.705160 8366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:09.706197 8366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:09.707828 8366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:09.707904 8366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:09.708087 8366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:09.708180 8366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:09.743125 8366 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:34041
I20260810 11:49:09.743177 8504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:34041 every 8 connection(s)
I20260810 11:49:09.744474 8366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:09.753186 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8366
I20260810 11:49:09.753401 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:09.753928 20854 external_mini_cluster.cc:1428] Setting key c5fee994eb5ada9d24b79a20963bdddf
Aug 10 11:49:09 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362549, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:09.766844 8507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:09.747249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:39909 (local address 127.20.93.190:42881)
0810 11:49:09.747746 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:09.747753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:09.748731 (+ 978us) server_negotiation.cc:407] Connection header received
0810 11:49:09.750665 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:09.750676 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:09.750751 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:09.750853 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:09.752354 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.753090 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.755161 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.755663 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.755795 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:09.759722 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:09.759731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:09.759738 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:09.759788 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:09.763356 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:09.764128 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:09.764137 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:09.764144 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:09.764304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:09.764740 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:09.764748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:09.764754 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:09.765146 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:09.765306 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:09.766086 (+ 780us) server_negotiation.cc:299] Negotiation successful
0810 11:49:09.766368 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":232,"threads_started":1}
I20260810 11:49:09.767630 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:09.747857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42881 (local address 127.20.93.129:39909)
0810 11:49:09.748554 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:09.748599 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:09.750386 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:09.751070 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:09.751078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:09.751627 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:09.752196 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:09.752212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.753274 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.753278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:09.754981 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:09.754998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:09.755916 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:09.755924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:09.756167 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:09.756902 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:09.756929 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:09.759463 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:09.763520 (+ 4057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:09.763528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:09.763542 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:09.763984 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:09.764415 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:09.764418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:09.764420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:09.764615 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:09.765270 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:09.765275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:09.765666 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 11:49:09.765903 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 11:49:09.766324 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":279,"threads_started":1}
I20260810 11:49:09.769307 8505 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42881
I20260810 11:49:09.769692 8505 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:09.770442 8505 heartbeater.cc:511] Master 127.20.93.190:42881 requested a full tablet report, sending...
I20260810 11:49:09.772610 8300 ts_manager.cc:194] Registered new tserver with Master: 914c7ec8d90446beb82764e47381af3a (127.20.93.129:34041)
I20260810 11:49:09.774196 8300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:39909
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:49:09.825976 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:10.180751 8511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:10.181033 8511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:10.181236 8511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:10.188591 8511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:10.188733 8511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:10.188927 8511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:10.199129 8511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:10.200560 8511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:10.202025 8511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:10.216506 8516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:10.216697 8519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:10.216729 8517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:10.218019 8511 server_base.cc:1034] running on GCE node
I20260810 11:49:10.218837 8511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:10.220099 8511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:10.221354 8511 hybrid_clock.cc:648] HybridClock initialized: now 1786362550221298 us; error 54 us; skew 500 ppm
I20260810 11:49:10.221782 8511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:10 dist-test-slave-njxd krb5kdc[8250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:10.225101 8511 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:49:10.226624 8511 webserver.cc:460] Webserver started at http://127.20.93.130:33285/ using document root <none> and password file <none>
I20260810 11:49:10.227315 8511 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:10.227418 8511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:10.227777 8511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:10.230894 8511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "567344e239bd4bbbb354405a052fd54e"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "d22b01d68a1efd750ca8a083581fa388"
server_key_iv: "2d593d17493ab2c265e8e01ab9e558d2"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.231635 8511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "567344e239bd4bbbb354405a052fd54e"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "d22b01d68a1efd750ca8a083581fa388"
server_key_iv: "2d593d17493ab2c265e8e01ab9e558d2"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.237253 8511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:49:10.241344 8527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.242913 8511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:10.243108 8511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "567344e239bd4bbbb354405a052fd54e"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "d22b01d68a1efd750ca8a083581fa388"
server_key_iv: "2d593d17493ab2c265e8e01ab9e558d2"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.243340 8511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:10.273507 8511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:10.278412 8511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:10.278789 8511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:10.279930 8511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:10.281471 8511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:10.281550 8511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.281744 8511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:10.281807 8511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.318224 8511 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:36371
I20260810 11:49:10.318312 8649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:36371 every 8 connection(s)
I20260810 11:49:10.319795 8511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:10.329946 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8511
I20260810 11:49:10.330155 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:10.330749 20854 external_mini_cluster.cc:1428] Setting key f8012bfca034d75f26828aa9723589a2
Aug 10 11:49:10 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:10.343998 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.322711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:51049 (local address 127.20.93.190:42881)
0810 11:49:10.323190 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.323195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.324308 (+ 1113us) server_negotiation.cc:407] Connection header received
0810 11:49:10.326332 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.326338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.326442 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.326571 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:10.328251 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.329007 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.331459 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.332078 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.332234 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.336426 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:10.336435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:10.336443 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:10.336495 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:10.340289 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.341160 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.341169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.341175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.341351 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.341862 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.341870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.341875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.342279 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:10.342443 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.343218 (+ 775us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.343492 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":241,"threads_started":1}
I20260810 11:49:10.344866 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.323375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42881 (local address 127.20.93.130:51049)
0810 11:49:10.324115 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:10.324162 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:10.326023 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:10.326846 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:10.326855 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:10.327415 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:10.328051 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.328069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.329177 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.329182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.331280 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.331297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.332347 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.332355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.332606 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.333487 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:10.333513 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:10.336145 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:10.340477 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:10.340485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:10.340500 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:10.341003 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:10.341473 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:10.341476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:10.341479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:10.341734 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:10.342418 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:10.342424 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:10.342805 (+ 381us) client_negotiation.cc:769] Sending connection context
0810 11:49:10.343066 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 11:49:10.343508 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260810 11:49:10.346611 8651 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42881
I20260810 11:49:10.346969 8651 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:10.347906 8651 heartbeater.cc:511] Master 127.20.93.190:42881 requested a full tablet report, sending...
I20260810 11:49:10.349732 8300 ts_manager.cc:194] Registered new tserver with Master: 567344e239bd4bbbb354405a052fd54e (127.20.93.130:36371)
I20260810 11:49:10.350612 8300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:51049
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:49:10.405531 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:10.767719 8657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:10.768167 8657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:10.768347 8657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:10.776322 8657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:10.776453 8657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:10.776639 8657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:10.777176 8505 heartbeater.cc:503] Master 127.20.93.190:42881 was elected leader, sending a full tablet report...
I20260810 11:49:10.787140 8657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:10.788641 8657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:10.790172 8657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:10.804668 8666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:10.804818 8662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:10.804845 8663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:10.805902 8657 server_base.cc:1034] running on GCE node
I20260810 11:49:10.806735 8657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:10.807869 8657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:10.809128 8657 hybrid_clock.cc:648] HybridClock initialized: now 1786362550809043 us; error 77 us; skew 500 ppm
I20260810 11:49:10.809485 8657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:10 dist-test-slave-njxd krb5kdc[8250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:10.812811 8657 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:49:10.814459 8657 webserver.cc:460] Webserver started at http://127.20.93.131:40391/ using document root <none> and password file <none>
I20260810 11:49:10.815187 8657 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:10.815340 8657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:10.815677 8657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:10.818727 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "44814e17e63443f782a20a865381f7b4"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "1bca7aa406e21f8aaa347f0f767f2b0b"
server_key_iv: "65bd4b5de46fec64a9cec853a4690587"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.819526 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "44814e17e63443f782a20a865381f7b4"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "1bca7aa406e21f8aaa347f0f767f2b0b"
server_key_iv: "65bd4b5de46fec64a9cec853a4690587"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.824890 8657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:10.828588 8673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.830133 8657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:10.830314 8657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "44814e17e63443f782a20a865381f7b4"
format_stamp: "Formatted at 2026-08-10 11:49:10 on dist-test-slave-njxd"
server_key: "1bca7aa406e21f8aaa347f0f767f2b0b"
server_key_iv: "65bd4b5de46fec64a9cec853a4690587"
server_key_version: "encryptionkey@0"
I20260810 11:49:10.830518 8657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:10.854499 8657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:10.859076 8657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:10.859391 8657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:10.860424 8657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:10.861887 8657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:10.861996 8657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.862159 8657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:10.862236 8657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:10.896458 8657 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:42099
I20260810 11:49:10.896518 8793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:42099 every 8 connection(s)
I20260810 11:49:10.897889 8657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:10.899395 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8657
I20260810 11:49:10.899595 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:10.900103 20854 external_mini_cluster.cc:1428] Setting key 31e0508e2cc835a0801e55255c550121
Aug 10 11:49:10 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362550, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:10.918849 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.900643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:57321 (local address 127.20.93.190:42881)
0810 11:49:10.901394 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.901402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.902479 (+ 1077us) server_negotiation.cc:407] Connection header received
0810 11:49:10.904542 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.904547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.904663 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.904756 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:10.906317 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.907250 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.908872 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.909271 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.909363 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.912788 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:10.912792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:10.912796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:10.912829 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:10.916111 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.916777 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.916781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.916783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.916898 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.917275 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.917278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.917280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.917545 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:10.917719 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.918410 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.918576 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":464,"threads_started":1}
I20260810 11:49:10.919862 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.901357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:42881 (local address 127.20.93.131:57321)
0810 11:49:10.902302 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:10.902362 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:10.904267 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:10.904965 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:10.904973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:10.905460 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:10.906144 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.906163 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.907445 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.907451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.908748 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.908761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.909468 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.909473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.909760 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.910372 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:10.910394 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:10.912590 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:10.916226 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:10.916232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:10.916245 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:10.916670 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:10.916985 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:10.916988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:10.916990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:10.917176 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:10.917713 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:10.917719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:10.918064 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 11:49:10.918312 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 11:49:10.918694 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":370,"threads_started":1}
I20260810 11:49:10.921221 8794 heartbeater.cc:347] Connected to a master server at 127.20.93.190:42881
I20260810 11:49:10.921500 8794 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:10.922252 8794 heartbeater.cc:511] Master 127.20.93.190:42881 requested a full tablet report, sending...
I20260810 11:49:10.923597 8300 ts_manager.cc:194] Registered new tserver with Master: 44814e17e63443f782a20a865381f7b4 (127.20.93.131:42099)
I20260810 11:49:10.924363 8300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:57321
I20260810 11:49:10.928045 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:10.945199 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.932213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.20.93.190:42881)
0810 11:49:10.932371 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.932375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.932895 (+ 520us) server_negotiation.cc:407] Connection header received
0810 11:49:10.933031 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.933035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.933106 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.933198 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:10.934666 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.935555 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.936875 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.937254 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.937348 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.940227 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:10.940232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:10.940235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:10.940265 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:10.942773 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.943522 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.943526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.943528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.943639 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:10.944011 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:10.944014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:10.944016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:10.944307 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:10.944493 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.944841 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.944970 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:49:10.949373 8300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:10.956256 8300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:10.983507 8811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.968832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.20.93.131:42099)
0810 11:49:10.975194 (+ 6362us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.975201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.975229 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:49:10.975302 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.975307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.975491 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.976560 (+ 1069us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:10.976849 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.978132 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.980975 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.982265 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.982416 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.982689 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.982961 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.983131 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6196,"thread_start_us":6065,"threads_started":1}
I20260810 11:49:10.983594 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.968521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:42099 (local address 127.0.0.1:35392)
0810 11:49:10.973750 (+ 5229us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:10.973810 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:10.974023 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:10.975782 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:10.975790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:10.975856 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:10.976337 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.976351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.978292 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.978299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.980797 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.980811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.982483 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.982490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.982722 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.982778 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 11:49:10.982918 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 11:49:10.983114 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4862,"mutex_wait_us":1,"spinlock_wait_cycles":4992,"thread_start_us":6776,"threads_started":1}
I20260810 11:49:10.984659 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.968935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:34041 (local address 127.0.0.1:46452)
0810 11:49:10.975589 (+ 6654us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:10.975616 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:10.975836 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:10.976894 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:10.976902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:10.976936 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:10.977404 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.977421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.979329 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.979336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.981651 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.981670 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.983816 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.983824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.984011 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.984045 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:49:10.984129 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 11:49:10.984265 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6512,"mutex_wait_us":5,"thread_start_us":4284,"threads_started":1}
I20260810 11:49:10.985530 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.969216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.20.93.129:34041)
0810 11:49:10.974710 (+ 5494us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.974719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.975719 (+ 1000us) server_negotiation.cc:407] Connection header received
0810 11:49:10.976062 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.976070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.976353 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.976546 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:10.977618 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.979157 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.981819 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.983284 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.983413 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.983661 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.985007 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.985194 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5296,"thread_start_us":5359,"threads_started":1}
I20260810 11:49:10.986994 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.977185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:36371 (local address 127.0.0.1:42258)
0810 11:49:10.978049 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:10.978078 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:10.978208 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:10.979152 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:10.979157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:10.979183 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:10.979797 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.979809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.981541 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.981548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.983714 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:10.983728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.986184 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.986190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:10.986355 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.986385 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:49:10.986487 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 11:49:10.986617 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":106,"thread_start_us":540,"threads_started":1}
I20260810 11:49:10.987044 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:10.975972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.20.93.130:36371)
0810 11:49:10.978719 (+ 2747us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:10.978725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:10.978745 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:49:10.978823 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:10.978828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:10.978973 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:10.979167 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:10.980049 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.981402 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.983923 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:10.985709 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:10.985843 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:10.986046 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:10.986563 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 11:49:10.986743 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2609,"thread_start_us":2479,"threads_started":1}
I20260810 11:49:10.987993 8726 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (DEFAULT_TABLE table=test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:10.989606 8726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:10.989293 8431 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (DEFAULT_TABLE table=test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:10.990617 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:10.991073 8580 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (DEFAULT_TABLE table=test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:10.992658 8580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:11.002686 8815 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Bootstrap starting.
I20260810 11:49:11.005256 8816 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Bootstrap starting.
I20260810 11:49:11.007213 8815 tablet_bootstrap.cc:654] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:11.009142 8816 tablet_bootstrap.cc:654] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:11.009219 8815 log.cc:824] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:11.010547 8816 log.cc:824] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:11.011965 8817 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Bootstrap starting.
I20260810 11:49:11.012717 8815 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: No bootstrap required, opened a new log
I20260810 11:49:11.013005 8815 ts_tablet_manager.cc:1397] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 11:49:11.013707 8816 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: No bootstrap required, opened a new log
I20260810 11:49:11.013967 8816 ts_tablet_manager.cc:1397] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 11:49:11.015000 8817 tablet_bootstrap.cc:654] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:11.015931 8817 log.cc:824] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:11.018383 8817 tablet_bootstrap.cc:492] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: No bootstrap required, opened a new log
I20260810 11:49:11.018637 8817 ts_tablet_manager.cc:1397] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 11:49:11.023933 8815 raft_consensus.cc:348] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.024420 8815 raft_consensus.cc:374] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:11.024581 8815 raft_consensus.cc:729] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914c7ec8d90446beb82764e47381af3a, State: Initialized, Role: FOLLOWER
I20260810 11:49:11.024576 8816 raft_consensus.cc:348] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.025085 8816 raft_consensus.cc:374] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:11.025190 8816 raft_consensus.cc:729] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44814e17e63443f782a20a865381f7b4, State: Initialized, Role: FOLLOWER
I20260810 11:49:11.025245 8815 consensus_queue.cc:260] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.025805 8816 consensus_queue.cc:260] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.026727 8817 raft_consensus.cc:348] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.027068 8817 raft_consensus.cc:374] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:11.027186 8817 raft_consensus.cc:729] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 567344e239bd4bbbb354405a052fd54e, State: Initialized, Role: FOLLOWER
I20260810 11:49:11.027331 8815 ts_tablet_manager.cc:1428] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 11:49:11.027777 8817 consensus_queue.cc:260] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.028174 8794 heartbeater.cc:503] Master 127.20.93.190:42881 was elected leader, sending a full tablet report...
I20260810 11:49:11.028971 8816 ts_tablet_manager.cc:1428] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260810 11:49:11.030073 8651 heartbeater.cc:503] Master 127.20.93.190:42881 was elected leader, sending a full tablet report...
I20260810 11:49:11.030711 8817 ts_tablet_manager.cc:1428] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
W20260810 11:49:11.072105 8652 tablet.cc:2369] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:11.085268 8824 raft_consensus.cc:482] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:11.085631 8824 raft_consensus.cc:504] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.087531 8824 leader_election.cc:288] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44814e17e63443f782a20a865381f7b4 (127.20.93.131:42099), 914c7ec8d90446beb82764e47381af3a (127.20.93.129:34041)
I20260810 11:49:11.094738 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.088192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:38601 (local address 127.20.93.129:34041)
0810 11:49:11.088432 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.088441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.089192 (+ 751us) server_negotiation.cc:407] Connection header received
0810 11:49:11.089374 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.089379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.089465 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.089561 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:11.090250 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.090986 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.092858 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.093747 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.093915 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.094003 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.094329 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.094468 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:49:11.094841 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.088140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:34041 (local address 127.20.93.130:38601)
0810 11:49:11.089090 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:11.089105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:11.089247 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:11.089619 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:11.089624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:11.089648 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:11.090060 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:11.090093 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.091134 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.091138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:11.092688 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:11.092703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.093977 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.093985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:11.094156 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.094185 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 11:49:11.094290 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 11:49:11.094413 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":38,"thread_start_us":567,"threads_started":1}
I20260810 11:49:11.094967 8811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.087905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:47083 (local address 127.20.93.131:42099)
0810 11:49:11.088137 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.088143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.089032 (+ 889us) server_negotiation.cc:407] Connection header received
0810 11:49:11.089295 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.089302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.089388 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.089454 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:11.090151 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.090909 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.092887 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.093834 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.093999 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.094073 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.094579 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.094707 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:49:11.095603 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.087967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:42099 (local address 127.20.93.130:47083)
0810 11:49:11.088903 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:11.088919 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:11.089108 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:11.089501 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:11.089509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:11.089606 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:11.089998 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:11.090009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.091042 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.091051 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:11.092716 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:11.092728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.094196 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.094202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:11.094364 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.094396 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:49:11.095163 (+ 767us) client_negotiation.cc:240] Negotiation successful
0810 11:49:11.095331 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":67,"thread_start_us":499,"threads_started":1}
I20260810 11:49:11.095999 8456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb5dbdf769f44e89e56e4029cc9c0ff" candidate_uuid: "567344e239bd4bbbb354405a052fd54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "914c7ec8d90446beb82764e47381af3a" is_pre_election: true
I20260810 11:49:11.096621 8456 raft_consensus.cc:2454] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 567344e239bd4bbbb354405a052fd54e in term 0.
I20260810 11:49:11.096611 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb5dbdf769f44e89e56e4029cc9c0ff" candidate_uuid: "567344e239bd4bbbb354405a052fd54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44814e17e63443f782a20a865381f7b4" is_pre_election: true
I20260810 11:49:11.097206 8746 raft_consensus.cc:2454] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 567344e239bd4bbbb354405a052fd54e in term 0.
I20260810 11:49:11.097414 8530 leader_election.cc:302] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 567344e239bd4bbbb354405a052fd54e, 914c7ec8d90446beb82764e47381af3a; no voters:
I20260810 11:49:11.097872 8824 raft_consensus.cc:2781] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:11.098044 8824 raft_consensus.cc:482] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:11.098161 8824 raft_consensus.cc:3037] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:11.100389 8824 raft_consensus.cc:504] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.100992 8824 leader_election.cc:288] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [CANDIDATE]: Term 1 election: Requested vote from peers 44814e17e63443f782a20a865381f7b4 (127.20.93.131:42099), 914c7ec8d90446beb82764e47381af3a (127.20.93.129:34041)
I20260810 11:49:11.101387 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb5dbdf769f44e89e56e4029cc9c0ff" candidate_uuid: "567344e239bd4bbbb354405a052fd54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44814e17e63443f782a20a865381f7b4"
I20260810 11:49:11.101493 8456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb5dbdf769f44e89e56e4029cc9c0ff" candidate_uuid: "567344e239bd4bbbb354405a052fd54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "914c7ec8d90446beb82764e47381af3a"
I20260810 11:49:11.101619 8746 raft_consensus.cc:3037] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:11.101774 8456 raft_consensus.cc:3037] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:11.104033 8746 raft_consensus.cc:2454] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 567344e239bd4bbbb354405a052fd54e in term 1.
I20260810 11:49:11.104108 8456 raft_consensus.cc:2454] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 567344e239bd4bbbb354405a052fd54e in term 1.
I20260810 11:49:11.104458 8529 leader_election.cc:302] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44814e17e63443f782a20a865381f7b4, 567344e239bd4bbbb354405a052fd54e; no voters:
I20260810 11:49:11.104790 8824 raft_consensus.cc:2781] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:11.105230 8824 raft_consensus.cc:686] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 1 LEADER]: Becoming Leader. State: Replica: 567344e239bd4bbbb354405a052fd54e, State: Running, Role: LEADER
I20260810 11:49:11.105775 8824 consensus_queue.cc:237] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } }
I20260810 11:49:11.110893 8299 catalog_manager.cc:5495] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e reported cstate change: term changed from 0 to 1, leader changed from <none> to 567344e239bd4bbbb354405a052fd54e (127.20.93.130). New cstate: current_term: 1 leader_uuid: "567344e239bd4bbbb354405a052fd54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "567344e239bd4bbbb354405a052fd54e" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 36371 } health_report { overall_health: HEALTHY } } }
I20260810 11:49:11.137305 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.131866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.20.93.130:36371)
0810 11:49:11.132059 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.132064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.132215 (+ 151us) server_negotiation.cc:407] Connection header received
0810 11:49:11.132388 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.132392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.132469 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.132591 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:11.132959 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.133741 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.135296 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.135702 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.135798 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.136279 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:11.136779 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:11.136918 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.137051 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.137093 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:49:11.146003 8746 raft_consensus.cc:1261] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 1 FOLLOWER]: Refusing update from remote peer 567344e239bd4bbbb354405a052fd54e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:11.146049 8456 raft_consensus.cc:1261] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 1 FOLLOWER]: Refusing update from remote peer 567344e239bd4bbbb354405a052fd54e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:11.147014 8824 consensus_queue.cc:1035] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44814e17e63443f782a20a865381f7b4" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:11.147354 8827 consensus_queue.cc:1035] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "914c7ec8d90446beb82764e47381af3a" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 34041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:49:11.149605 8795 tablet.cc:2369] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:11.171512 8299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:11.171816 8299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:11.175473 8299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fed4d05c54b4e1db1bf98adec36ce97: Sending DeleteTablet for 3 replicas of tablet 2bb5dbdf769f44e89e56e4029cc9c0ff
I20260810 11:49:11.176582 8431 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:46452
I20260810 11:49:11.176671 8580 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:42258
I20260810 11:49:11.177613 8726 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb5dbdf769f44e89e56e4029cc9c0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:11 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:35392
I20260810 11:49:11.178184 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8366
I20260810 11:49:11.189112 8839 tablet_replica.cc:330] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: stopping tablet replica
I20260810 11:49:11.189810 8839 raft_consensus.cc:2229] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:11.190466 8839 raft_consensus.cc:2258] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:11.192775 8839 ts_tablet_manager.cc:1905] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:11.198098 8840 tablet_replica.cc:330] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: stopping tablet replica
I20260810 11:49:11.198669 8842 tablet_replica.cc:330] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: stopping tablet replica
I20260810 11:49:11.199265 8842 raft_consensus.cc:2229] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:11.199836 8842 raft_consensus.cc:2258] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:11.200511 8840 raft_consensus.cc:2229] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:11.200654 8839 ts_tablet_manager.cc:1918] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:11.200879 8839 log.cc:1196] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/2bb5dbdf769f44e89e56e4029cc9c0ff
I20260810 11:49:11.201566 8839 ts_tablet_manager.cc:1939] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 914c7ec8d90446beb82764e47381af3a: Deleting consensus metadata
I20260810 11:49:11.202256 8840 raft_consensus.cc:2258] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:11.202311 8842 ts_tablet_manager.cc:1905] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:11.203637 8284 catalog_manager.cc:4841] TS 914c7ec8d90446beb82764e47381af3a (127.20.93.129:34041): tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (table test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]) successfully deleted
I20260810 11:49:11.205662 8840 ts_tablet_manager.cc:1905] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:49:11 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:49:11.210521 8842 ts_tablet_manager.cc:1918] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:11.210799 8842 log.cc:1196] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/2bb5dbdf769f44e89e56e4029cc9c0ff
I20260810 11:49:11.211444 8842 ts_tablet_manager.cc:1939] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 44814e17e63443f782a20a865381f7b4: Deleting consensus metadata
I20260810 11:49:11.213446 8283 catalog_manager.cc:4841] TS 44814e17e63443f782a20a865381f7b4 (127.20.93.131:42099): tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (table test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]) successfully deleted
I20260810 11:49:11.213703 8840 ts_tablet_manager.cc:1918] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:11.214001 8840 log.cc:1196] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/2bb5dbdf769f44e89e56e4029cc9c0ff
I20260810 11:49:11.214579 8840 ts_tablet_manager.cc:1939] T 2bb5dbdf769f44e89e56e4029cc9c0ff P 567344e239bd4bbbb354405a052fd54e: Deleting consensus metadata
I20260810 11:49:11.216008 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.179879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.20.93.129:34041)
0810 11:49:11.180109 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.180119 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.192020 (+ 11901us) server_negotiation.cc:407] Connection header received
0810 11:49:11.192483 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.192492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.192579 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.192653 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:11.194641 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.195765 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.198310 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.198966 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.199089 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.207566 (+ 8477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:11.207582 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:11.207602 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:11.207646 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:11.211796 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.213269 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.213280 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.213286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.213524 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.214168 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.214182 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.214185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.214691 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:11.215065 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.215354 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.215608 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:49:11.216388 8283 catalog_manager.cc:4841] TS 567344e239bd4bbbb354405a052fd54e (127.20.93.130:36371): tablet 2bb5dbdf769f44e89e56e4029cc9c0ff (table test-table [id=fbd26f18a3014618bcbc5cb251c53ac4]) successfully deleted
I20260810 11:49:11.217060 8499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:11.310021 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8366
I20260810 11:49:11.338321 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8511
Aug 10 11:49:11 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:49:11.359290 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.339183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.20.93.130:36371)
0810 11:49:11.339391 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.339398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.339490 (+ 92us) server_negotiation.cc:407] Connection header received
0810 11:49:11.339664 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.339670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.339756 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.339869 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:11.341496 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.342876 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.344833 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.345562 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.345729 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.351903 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:11.351920 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:11.351939 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:11.351986 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:11.356236 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.357179 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.357186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.357188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.357345 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.357807 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.357810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.357812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.358179 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:11.358445 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.358796 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.358968 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:49:11.360040 8644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:11.477990 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8511
I20260810 11:49:11.500379 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8657
Aug 10 11:49:11 dist-test-slave-njxd krb5kdc[8250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:49:11.517277 8811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:11.501240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.20.93.131:42099)
0810 11:49:11.501506 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:11.501511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:11.501534 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:49:11.501812 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:11.501817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:11.501890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:11.502009 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:11.503584 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.504454 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.505848 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:11.506363 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:11.506468 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:11.511687 (+ 5219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:11.511706 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:11.511723 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:11.511766 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:11.514708 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.515423 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.515432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.515434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.515608 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:11.515954 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:11.515957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:11.515959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:11.516345 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:11.516606 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:11.516846 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 11:49:11.517031 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:49:11.518011 8787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:11.610666 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8657
I20260810 11:49:11.632778 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8266
I20260810 11:49:11.634186 8332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:11.756345 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8266
2026-08-10T11:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4690 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T11:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:11Z Disabled control of system clock
I20260810 11:49:11.855196 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38241
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:12.212671 8853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:12.212987 8853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:12.213071 8853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:12.220383 8853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:12.220502 8853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:12.220542 8853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:12.220599 8853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:12.220642 8853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:12.229998 8853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38241
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:12.231333 8853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:12.233345 8853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:12.246212 8858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:12.246264 8861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:12.246170 8859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:12.247364 8853 server_base.cc:1034] running on GCE node
I20260810 11:49:12.248322 8853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:12.249967 8853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:12.251258 8853 hybrid_clock.cc:648] HybridClock initialized: now 1786362552251194 us; error 58 us; skew 500 ppm
I20260810 11:49:12.251700 8853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:12.254467 8853 webserver.cc:460] Webserver started at http://127.20.93.190:36189/ using document root <none> and password file <none>
I20260810 11:49:12.255285 8853 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:12.255435 8853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:12.255795 8853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:12.259191 8853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "44682be914394f0dac1f39d22d3ebfa3"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "c1bb538a509355e7e60f7ba0c4fee1ee"
server_key_iv: "67ab09f7b0fa59692bba7d30d1dc2451"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.260119 8853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "44682be914394f0dac1f39d22d3ebfa3"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "c1bb538a509355e7e60f7ba0c4fee1ee"
server_key_iv: "67ab09f7b0fa59692bba7d30d1dc2451"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.266065 8853 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:49:12.270398 8867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:12.272083 8853 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:49:12.272356 8853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "44682be914394f0dac1f39d22d3ebfa3"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "c1bb538a509355e7e60f7ba0c4fee1ee"
server_key_iv: "67ab09f7b0fa59692bba7d30d1dc2451"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.272625 8853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:12.298171 8853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:12.299167 8853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:12.299502 8853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:12.319170 8853 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38241
I20260810 11:49:12.319195 8920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38241 every 8 connection(s)
I20260810 11:49:12.320640 8853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:12.324834 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:12.328504 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8853
I20260810 11:49:12.328739 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:12.329200 20854 external_mini_cluster.cc:1428] Setting key eb9179a07ab97fcdcc25518aeed4cbc4
I20260810 11:49:12.339659 8921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: Bootstrap starting.
I20260810 11:49:12.343911 8921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:12.345395 8921 log.cc:824] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:12.346315 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:12.331403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.20.93.190:38241)
0810 11:49:12.332598 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:12.332626 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:12.332683 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:49:12.334756 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:12.334793 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:12.335467 (+ 674us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:12.335970 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:12.336593 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.338680 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.340458 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.341619 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.341836 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:12.343036 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:12.343046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:12.343067 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:12.343829 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:12.343978 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:12.344008 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:12.344125 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:12.344455 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 11:49:12.344835 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":392,"threads_started":1}
I20260810 11:49:12.349100 8921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: No bootstrap required, opened a new log
I20260810 11:49:12.356788 8921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } }
I20260810 11:49:12.357209 8921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:12.357296 8921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44682be914394f0dac1f39d22d3ebfa3, State: Initialized, Role: FOLLOWER
I20260810 11:49:12.357913 8921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } }
I20260810 11:49:12.358135 8921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:12.358242 8921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:12.358369 8921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:12.360844 8921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } }
I20260810 11:49:12.361337 8921 leader_election.cc:302] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44682be914394f0dac1f39d22d3ebfa3; no voters:
I20260810 11:49:12.361980 8921 leader_election.cc:288] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:12.362181 8927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:12.363098 8927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [term 1 LEADER]: Becoming Leader. State: Replica: 44682be914394f0dac1f39d22d3ebfa3, State: Running, Role: LEADER
I20260810 11:49:12.363720 8927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } }
I20260810 11:49:12.364423 8921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:12.366802 8929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44682be914394f0dac1f39d22d3ebfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } } }
I20260810 11:49:12.366911 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44682be914394f0dac1f39d22d3ebfa3. Latest consensus state: current_term: 1 leader_uuid: "44682be914394f0dac1f39d22d3ebfa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44682be914394f0dac1f39d22d3ebfa3" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38241 } } }
I20260810 11:49:12.367262 8929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:12.367343 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:12.369757 8934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:12.377262 8934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:12.388770 8934 catalog_manager.cc:1347] Generated new cluster ID: 6b95fb2eb9dc4667920b8e223ffa4bff
I20260810 11:49:12.388912 8934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:12.405874 8934 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:12.407136 8934 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:12.414458 8934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: Generated new TSK 0
I20260810 11:49:12.415323 8934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:12.421131 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--builtin_ntp_servers=127.20.93.148:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:12.783852 8948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:12.784154 8948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:12.784318 8948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:12.791800 8948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:12.791908 8948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:12.792121 8948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:12.801607 8948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:12.803026 8948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:12.805055 8948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:12.819325 8957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:12.819370 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:12.819334 8955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:12.820521 8948 server_base.cc:1034] running on GCE node
I20260810 11:49:12.821165 8948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:12.822266 8948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:12.823571 8948 hybrid_clock.cc:648] HybridClock initialized: now 1786362552823488 us; error 41 us; skew 500 ppm
I20260810 11:49:12.823982 8948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:12.826573 8948 webserver.cc:460] Webserver started at http://127.20.93.129:37803/ using document root <none> and password file <none>
I20260810 11:49:12.827325 8948 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:12.827471 8948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:12.827812 8948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:12.831045 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "4fcf581e6c5c4297bdd23a046bc7c148"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "5efa52a9706f27dad048445087d44a03"
server_key_iv: "40bd451261c20f98d8c2a45b32ad3ba5"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.831897 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "4fcf581e6c5c4297bdd23a046bc7c148"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "5efa52a9706f27dad048445087d44a03"
server_key_iv: "40bd451261c20f98d8c2a45b32ad3ba5"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.837302 8948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:49:12.841274 8963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:12.842742 8948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 11:49:12.842937 8948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "4fcf581e6c5c4297bdd23a046bc7c148"
format_stamp: "Formatted at 2026-08-10 11:49:12 on dist-test-slave-njxd"
server_key: "5efa52a9706f27dad048445087d44a03"
server_key_iv: "40bd451261c20f98d8c2a45b32ad3ba5"
server_key_version: "encryptionkey@0"
I20260810 11:49:12.843190 8948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:12.867331 8948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:12.868238 8948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:12.868566 8948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:12.869674 8948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:12.871158 8948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:12.871289 8948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:12.871378 8948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:12.871435 8948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:12.905292 8948 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:40813
I20260810 11:49:12.905397 9087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:40813 every 8 connection(s)
I20260810 11:49:12.906682 8948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:12.909884 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 8948
I20260810 11:49:12.910073 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:12.910776 20854 external_mini_cluster.cc:1428] Setting key 74d078835a450df0fa626e7aadfe6029
I20260810 11:49:12.913956 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--builtin_ntp_servers=127.20.93.148:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:12.921661 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:12.909198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:38341 (local address 127.20.93.190:38241)
0810 11:49:12.909692 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:12.909699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:12.911332 (+ 1633us) server_negotiation.cc:407] Connection header received
0810 11:49:12.913728 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:12.913732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:12.913796 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:12.913889 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:12.915429 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.916119 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.918646 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.919069 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.919164 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:12.920815 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:12.920820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:12.920824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:12.920842 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:12.920888 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:12.920900 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:12.920969 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:12.921240 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 11:49:12.921325 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":211,"threads_started":1}
I20260810 11:49:12.923208 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:12.909928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38241 (local address 127.20.93.129:38341)
0810 11:49:12.911126 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:12.911177 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:12.913393 (+ 2216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:12.914228 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:12.914244 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:12.914382 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:12.915245 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:12.915272 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.916288 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.916296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:12.918477 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:12.918496 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:12.919300 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:12.919310 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:12.919655 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:12.920422 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:12.920428 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:12.920487 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:12.920605 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:12.920998 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:12.921013 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:12.921025 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 11:49:12.921226 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 11:49:12.921714 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":343,"threads_started":1}
I20260810 11:49:12.924803 9088 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38241
I20260810 11:49:12.925159 9088 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:12.926153 9088 heartbeater.cc:511] Master 127.20.93.190:38241 requested a full tablet report, sending...
I20260810 11:49:12.928746 8885 ts_manager.cc:194] Registered new tserver with Master: 4fcf581e6c5c4297bdd23a046bc7c148 (127.20.93.129:40813)
I20260810 11:49:12.931071 8885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.129:38341
W20260810 11:49:13.274782 9092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:13.275059 9092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:13.275147 9092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:13.282522 9092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:13.282625 9092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:13.282732 9092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:13.291642 9092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:13.292943 9092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:13.294754 9092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:13.307500 9097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:13.307617 9100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:13.307621 9098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:13.308576 9092 server_base.cc:1034] running on GCE node
I20260810 11:49:13.309206 9092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:13.310181 9092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:13.311409 9092 hybrid_clock.cc:648] HybridClock initialized: now 1786362553311366 us; error 57 us; skew 500 ppm
I20260810 11:49:13.311820 9092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:13.314018 9092 webserver.cc:460] Webserver started at http://127.20.93.130:44605/ using document root <none> and password file <none>
I20260810 11:49:13.314730 9092 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:13.314860 9092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:13.315169 9092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:13.317961 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "85e53d26ef0d462fb74c86ee023ee237"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "95b7a74464f848242ae30c20058df080"
server_key_iv: "1ce67360bdaced61dae45927cc9a026f"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.318756 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "85e53d26ef0d462fb74c86ee023ee237"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "95b7a74464f848242ae30c20058df080"
server_key_iv: "1ce67360bdaced61dae45927cc9a026f"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.323800 9092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:13.327380 9106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.328740 9092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:49:13.328917 9092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "85e53d26ef0d462fb74c86ee023ee237"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "95b7a74464f848242ae30c20058df080"
server_key_iv: "1ce67360bdaced61dae45927cc9a026f"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.329138 9092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:13.349033 9092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:13.349944 9092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:13.350256 9092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:13.351267 9092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:13.352808 9092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:13.352932 9092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.353083 9092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:13.353156 9092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.387533 9092 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:46491
I20260810 11:49:13.387600 9229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:46491 every 8 connection(s)
I20260810 11:49:13.388862 9092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:13.394119 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9092
I20260810 11:49:13.394322 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:13.394797 20854 external_mini_cluster.cc:1428] Setting key bf9d8d6e4ed2620e00c9260a2fa7daaa
I20260810 11:49:13.397998 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--builtin_ntp_servers=127.20.93.148:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:13.402616 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.391418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:44723 (local address 127.20.93.190:38241)
0810 11:49:13.391634 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.391641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.393157 (+ 1516us) server_negotiation.cc:407] Connection header received
0810 11:49:13.395292 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.395299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.395395 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.395488 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:13.396666 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.397708 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.399429 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.400163 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.400315 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.401697 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:13.401705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:13.401712 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:13.401737 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:13.401779 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:13.401797 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:13.401889 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.402108 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.402228 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:49:13.403466 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.391936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38241 (local address 127.20.93.130:44723)
0810 11:49:13.392989 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:13.393036 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:13.394995 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:13.395778 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:13.395788 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:13.395882 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:13.396470 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.396487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.397863 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.397867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.399253 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.399266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.400394 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.400399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.400612 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.401342 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:13.401344 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:13.401384 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:13.401470 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:13.401895 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:13.401903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:13.401909 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:49:13.402070 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 11:49:13.402369 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":294,"threads_started":1}
I20260810 11:49:13.405092 9230 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38241
I20260810 11:49:13.405428 9230 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:13.406389 9230 heartbeater.cc:511] Master 127.20.93.190:38241 requested a full tablet report, sending...
I20260810 11:49:13.407936 8885 ts_manager.cc:194] Registered new tserver with Master: 85e53d26ef0d462fb74c86ee023ee237 (127.20.93.130:46491)
I20260810 11:49:13.409097 8885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.130:44723
W20260810 11:49:13.746341 9233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:13.746649 9233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:13.746749 9233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:13.754254 9233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:13.754367 9233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:13.754477 9233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:13.763708 9233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:38241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:13.765153 9233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:13.767000 9233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:13.780733 9242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:13.780694 9239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:13.780808 9238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:13.781471 9233 server_base.cc:1034] running on GCE node
I20260810 11:49:13.782141 9233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:13.783068 9233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:13.784300 9233 hybrid_clock.cc:648] HybridClock initialized: now 1786362553784261 us; error 54 us; skew 500 ppm
I20260810 11:49:13.784621 9233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:13.786947 9233 webserver.cc:460] Webserver started at http://127.20.93.131:42769/ using document root <none> and password file <none>
I20260810 11:49:13.787662 9233 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:13.787807 9233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:13.788130 9233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:13.791205 9233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "7b35e541350c4cc6956181a498034829"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "af5292a55002ea25706cd1d1c8c96286"
server_key_iv: "93df3cb44093be2ec6e3e216e02e0834"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.792063 9233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "7b35e541350c4cc6956181a498034829"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "af5292a55002ea25706cd1d1c8c96286"
server_key_iv: "93df3cb44093be2ec6e3e216e02e0834"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.797428 9233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 11:49:13.801111 9248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.802543 9233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:13.802742 9233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "7b35e541350c4cc6956181a498034829"
format_stamp: "Formatted at 2026-08-10 11:49:13 on dist-test-slave-njxd"
server_key: "af5292a55002ea25706cd1d1c8c96286"
server_key_iv: "93df3cb44093be2ec6e3e216e02e0834"
server_key_version: "encryptionkey@0"
I20260810 11:49:13.802978 9233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:13.826103 9233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:13.826993 9233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:13.827313 9233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:13.828294 9233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:13.829869 9233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:13.829994 9233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.830123 9233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:13.830212 9233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:13.864351 9233 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:32991
I20260810 11:49:13.864374 9372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:32991 every 8 connection(s)
I20260810 11:49:13.865895 9233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:13.871367 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9233
I20260810 11:49:13.871563 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:13.872175 20854 external_mini_cluster.cc:1428] Setting key 8578b88f7a28c00f5a46fbfbe2e348ac
I20260810 11:49:13.879629 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.868600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:51351 (local address 127.20.93.190:38241)
0810 11:49:13.868781 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.868787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.870212 (+ 1425us) server_negotiation.cc:407] Connection header received
0810 11:49:13.872827 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.872833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.872934 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.873101 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:13.874375 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.875134 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.876706 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.877190 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.877323 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.878754 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:13.878759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:13.878762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:13.878782 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:13.878814 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:13.878826 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:13.878918 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.879150 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.879260 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:49:13.880659 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.869042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:38241 (local address 127.20.93.131:51351)
0810 11:49:13.870041 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:13.870085 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:13.872470 (+ 2385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:13.873343 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:13.873352 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:13.873443 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:13.874208 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.874227 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.875301 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.875306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.876562 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.876574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.877422 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.877429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.877724 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.878387 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:13.878390 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:13.878430 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:13.878512 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:13.878917 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:13.878930 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:13.878940 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:49:13.879123 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 11:49:13.879525 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":289,"threads_started":1}
I20260810 11:49:13.881999 9373 heartbeater.cc:347] Connected to a master server at 127.20.93.190:38241
I20260810 11:49:13.882318 9373 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:13.883095 9373 heartbeater.cc:511] Master 127.20.93.190:38241 requested a full tablet report, sending...
I20260810 11:49:13.884445 8885 ts_manager.cc:194] Registered new tserver with Master: 7b35e541350c4cc6956181a498034829 (127.20.93.131:32991)
I20260810 11:49:13.885308 8885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.131:51351
I20260810 11:49:13.887460 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:13.897970 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.891648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.20.93.190:38241)
0810 11:49:13.891882 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.891891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.892229 (+ 338us) server_negotiation.cc:407] Connection header received
0810 11:49:13.892436 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.892441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.892510 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.892615 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:13.893121 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.893903 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.895323 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.895750 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.895886 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.897157 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:13.897162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:13.897165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:13.897183 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:13.897215 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:13.897228 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:13.897387 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.897622 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.897698 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:49:13.902015 8885 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:13.911237 8885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:13.934593 9088 heartbeater.cc:503] Master 127.20.93.190:38241 was elected leader, sending a full tablet report...
I20260810 11:49:13.936990 9391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.926214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:32991 (local address 127.0.0.1:45700)
0810 11:49:13.927895 (+ 1681us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:13.927912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:13.928025 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:13.929275 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:13.929278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:13.929298 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:13.929567 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.929626 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.931471 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.931476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.933787 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.933802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.935902 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.935913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.936152 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.936244 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 11:49:13.936378 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 11:49:13.936582 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1544,"mutex_wait_us":81,"thread_start_us":1351,"threads_started":1}
I20260810 11:49:13.937328 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.924662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.20.93.130:46491)
0810 11:49:13.925794 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.925806 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.926228 (+ 422us) server_negotiation.cc:407] Connection header received
0810 11:49:13.926519 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.926526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.926700 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.926853 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:13.928418 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.930075 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.933460 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.935322 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.935473 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.935719 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.936815 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.936987 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":784,"threads_started":1}
I20260810 11:49:13.937539 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.924247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46491 (local address 127.0.0.1:41882)
0810 11:49:13.926099 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:13.926119 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:13.926342 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:13.927133 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:13.927141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:13.927201 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:13.927734 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.927753 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.930222 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.930227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.933205 (+ 2978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.933220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.936212 (+ 2992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.936218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.936478 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.936613 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 11:49:13.936711 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 11:49:13.937129 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1692,"mutex_wait_us":594,"spinlock_wait_cycles":3968,"thread_start_us":1553,"threads_started":1}
I20260810 11:49:13.937892 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.926690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.20.93.131:32991)
0810 11:49:13.928468 (+ 1778us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.928476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.928508 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:49:13.928601 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.928610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.928861 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.929052 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:13.929799 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.931290 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.934007 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.935351 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.935488 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.935761 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.937276 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.937453 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1591,"thread_start_us":2052,"threads_started":1}
I20260810 11:49:13.937908 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.924251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.20.93.129:40813)
0810 11:49:13.926253 (+ 2002us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:13.926264 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:13.926290 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:49:13.926390 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:13.926401 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:13.926656 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:13.926867 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:13.928372 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.930095 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.933056 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.934636 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.934789 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.935099 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:13.936782 (+ 1683us) server_negotiation.cc:299] Negotiation successful
0810 11:49:13.936983 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1807,"thread_start_us":1708,"threads_started":1}
I20260810 11:49:13.937991 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:13.924001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36294)
0810 11:49:13.925114 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:13.925176 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:13.925380 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:13.927133 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:13.927141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:13.927182 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:13.928203 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.928213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.930277 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.930281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.932863 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:13.932875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:13.935860 (+ 2985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:13.935867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:13.936474 (+ 607us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:13.936610 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 11:49:13.936711 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 11:49:13.936858 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":1089,"spinlock_wait_cycles":768,"thread_start_us":622,"threads_started":1}
I20260810 11:49:13.941876 9019 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c83895714d9bbf51241b31058a2f (DEFAULT_TABLE table=test-table [id=109efd0b691e46e792fd412604274234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:13.941936 9299 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c83895714d9bbf51241b31058a2f (DEFAULT_TABLE table=test-table [id=109efd0b691e46e792fd412604274234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:13.941936 9157 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c83895714d9bbf51241b31058a2f (DEFAULT_TABLE table=test-table [id=109efd0b691e46e792fd412604274234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:13.943171 9019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c83895714d9bbf51241b31058a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:13.943295 9299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c83895714d9bbf51241b31058a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:13.943295 9157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c83895714d9bbf51241b31058a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:13.955435 9393 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Bootstrap starting.
I20260810 11:49:13.958151 9394 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Bootstrap starting.
I20260810 11:49:13.960330 9393 tablet_bootstrap.cc:654] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:13.961687 9393 log.cc:824] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:13.961928 9394 tablet_bootstrap.cc:654] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:13.962359 9395 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Bootstrap starting.
I20260810 11:49:13.963549 9394 log.cc:824] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:13.965508 9395 tablet_bootstrap.cc:654] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:13.966737 9395 log.cc:824] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:13.971647 9395 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: No bootstrap required, opened a new log
I20260810 11:49:13.971645 9393 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: No bootstrap required, opened a new log
I20260810 11:49:13.971958 9395 ts_tablet_manager.cc:1397] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 11:49:13.971959 9393 ts_tablet_manager.cc:1397] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.012s
I20260810 11:49:13.972429 9394 tablet_bootstrap.cc:492] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: No bootstrap required, opened a new log
I20260810 11:49:13.972710 9394 ts_tablet_manager.cc:1397] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260810 11:49:13.980882 9394 raft_consensus.cc:348] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.981316 9394 raft_consensus.cc:374] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:13.981454 9394 raft_consensus.cc:729] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e53d26ef0d462fb74c86ee023ee237, State: Initialized, Role: FOLLOWER
I20260810 11:49:13.982146 9394 consensus_queue.cc:260] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.983331 9230 heartbeater.cc:503] Master 127.20.93.190:38241 was elected leader, sending a full tablet report...
I20260810 11:49:13.983151 9393 raft_consensus.cc:348] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.983150 9395 raft_consensus.cc:348] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.983583 9395 raft_consensus.cc:374] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:13.983593 9393 raft_consensus.cc:374] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:13.983680 9395 raft_consensus.cc:729] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b35e541350c4cc6956181a498034829, State: Initialized, Role: FOLLOWER
I20260810 11:49:13.983685 9393 raft_consensus.cc:729] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcf581e6c5c4297bdd23a046bc7c148, State: Initialized, Role: FOLLOWER
I20260810 11:49:13.984043 9394 ts_tablet_manager.cc:1428] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260810 11:49:13.984306 9393 consensus_queue.cc:260] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.984306 9395 consensus_queue.cc:260] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:13.985453 9373 heartbeater.cc:503] Master 127.20.93.190:38241 was elected leader, sending a full tablet report...
I20260810 11:49:13.986204 9395 ts_tablet_manager.cc:1428] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 11:49:13.987783 9393 ts_tablet_manager.cc:1428] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260810 11:49:14.117745 9374 tablet.cc:2369] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:14.128545 9400 raft_consensus.cc:482] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:14.128854 9400 raft_consensus.cc:504] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:14.130851 9400 leader_election.cc:288] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fcf581e6c5c4297bdd23a046bc7c148 (127.20.93.129:40813), 85e53d26ef0d462fb74c86ee023ee237 (127.20.93.130:46491)
I20260810 11:49:14.138113 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.131113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:46491 (local address 127.20.93.131:35097)
0810 11:49:14.131419 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.131448 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:14.131620 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:14.132260 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:14.132270 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:14.132308 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:14.132524 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:14.132535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.133883 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.133886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:14.135965 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:14.135980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.137355 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.137360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:14.137485 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.137512 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:49:14.137724 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 11:49:14.137840 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":70}
I20260810 11:49:14.138186 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.131422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35097 (local address 127.20.93.130:46491)
0810 11:49:14.131644 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.131651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.131674 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:49:14.131751 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.131759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.131858 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.131959 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:14.132716 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.133714 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.136146 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.137173 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.137355 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.137431 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.137758 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.137901 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:49:14.138898 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.132441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:35357 (local address 127.20.93.129:40813)
0810 11:49:14.132675 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.132683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.132700 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:49:14.132772 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.132779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.132863 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.132969 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:14.133521 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.134347 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.136655 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.137934 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.138107 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.138171 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.138532 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.138664 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:49:14.138975 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.131261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.20.93.131:35357)
0810 11:49:14.132285 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.132309 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:14.132417 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:14.132988 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:14.132990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:14.133012 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:14.133311 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:14.133318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.134532 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.134538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:14.136463 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:14.136474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.138123 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.138133 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:14.138324 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.138366 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 11:49:14.138497 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 11:49:14.138617 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":3,"thread_start_us":676,"threads_started":1}
I20260810 11:49:14.139264 9184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c83895714d9bbf51241b31058a2f" candidate_uuid: "7b35e541350c4cc6956181a498034829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e53d26ef0d462fb74c86ee023ee237" is_pre_election: true
I20260810 11:49:14.139895 9184 raft_consensus.cc:2454] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b35e541350c4cc6956181a498034829 in term 0.
I20260810 11:49:14.140048 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c83895714d9bbf51241b31058a2f" candidate_uuid: "7b35e541350c4cc6956181a498034829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" is_pre_election: true
I20260810 11:49:14.140636 9040 raft_consensus.cc:2454] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b35e541350c4cc6956181a498034829 in term 0.
I20260810 11:49:14.140748 9249 leader_election.cc:302] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b35e541350c4cc6956181a498034829, 85e53d26ef0d462fb74c86ee023ee237; no voters:
W20260810 11:49:14.140892 9231 tablet.cc:2369] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:14.141245 9400 raft_consensus.cc:2781] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:14.141409 9400 raft_consensus.cc:482] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:14.141547 9400 raft_consensus.cc:3037] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:14.143865 9400 raft_consensus.cc:504] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:14.144517 9400 leader_election.cc:288] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [CANDIDATE]: Term 1 election: Requested vote from peers 4fcf581e6c5c4297bdd23a046bc7c148 (127.20.93.129:40813), 85e53d26ef0d462fb74c86ee023ee237 (127.20.93.130:46491)
I20260810 11:49:14.145123 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c83895714d9bbf51241b31058a2f" candidate_uuid: "7b35e541350c4cc6956181a498034829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcf581e6c5c4297bdd23a046bc7c148"
I20260810 11:49:14.145120 9184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c83895714d9bbf51241b31058a2f" candidate_uuid: "7b35e541350c4cc6956181a498034829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e53d26ef0d462fb74c86ee023ee237"
I20260810 11:49:14.145342 9184 raft_consensus.cc:3037] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:14.145342 9040 raft_consensus.cc:3037] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:14.148769 9184 raft_consensus.cc:2454] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b35e541350c4cc6956181a498034829 in term 1.
I20260810 11:49:14.148763 9040 raft_consensus.cc:2454] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b35e541350c4cc6956181a498034829 in term 1.
I20260810 11:49:14.149204 9249 leader_election.cc:302] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b35e541350c4cc6956181a498034829, 85e53d26ef0d462fb74c86ee023ee237; no voters:
I20260810 11:49:14.149550 9400 raft_consensus.cc:2781] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:14.150070 9400 raft_consensus.cc:686] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 1 LEADER]: Becoming Leader. State: Replica: 7b35e541350c4cc6956181a498034829, State: Running, Role: LEADER
I20260810 11:49:14.150581 9400 consensus_queue.cc:237] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } }
I20260810 11:49:14.155624 8884 catalog_manager.cc:5495] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b35e541350c4cc6956181a498034829 (127.20.93.131). New cstate: current_term: 1 leader_uuid: "7b35e541350c4cc6956181a498034829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b35e541350c4cc6956181a498034829" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 32991 } health_report { overall_health: HEALTHY } } }
W20260810 11:49:14.159227 9089 tablet.cc:2369] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:14.191151 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.185347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.20.93.131:32991)
0810 11:49:14.185536 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.185541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.185560 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:49:14.185843 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.185846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.185937 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.186069 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:14.186471 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.187233 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.188837 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.189275 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.189368 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.190004 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:14.190536 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:14.190669 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.190810 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.190848 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:49:14.199864 9040 raft_consensus.cc:1261] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 1 FOLLOWER]: Refusing update from remote peer 7b35e541350c4cc6956181a498034829: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:14.199864 9184 raft_consensus.cc:1261] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 1 FOLLOWER]: Refusing update from remote peer 7b35e541350c4cc6956181a498034829: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:14.201027 9403 consensus_queue.cc:1035] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e53d26ef0d462fb74c86ee023ee237" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:14.201416 9400 consensus_queue.cc:1035] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fcf581e6c5c4297bdd23a046bc7c148" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 40813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:14.213892 9407 mvcc.cc:205] Tried to move back new op lower bound from 7316941021990961152 to 7316941021804781568. Current Snapshot: MvccSnapshot[applied={T|T < 7316941021990961152}]
I20260810 11:49:14.227216 8883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:14.227567 8883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:14.231518 8883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44682be914394f0dac1f39d22d3ebfa3: Sending DeleteTablet for 3 replicas of tablet 9b16c83895714d9bbf51241b31058a2f
I20260810 11:49:14.232776 9019 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b16c83895714d9bbf51241b31058a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:14 UTC) from {username='slave'} at 127.0.0.1:36294
I20260810 11:49:14.232909 9157 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b16c83895714d9bbf51241b31058a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:14 UTC) from {username='slave'} at 127.0.0.1:41882
I20260810 11:49:14.233031 9299 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b16c83895714d9bbf51241b31058a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:14 UTC) from {username='slave'} at 127.0.0.1:45700
I20260810 11:49:14.234984 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8948
I20260810 11:49:14.243193 9419 tablet_replica.cc:330] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: stopping tablet replica
I20260810 11:49:14.243814 9419 raft_consensus.cc:2229] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:14.244649 9419 raft_consensus.cc:2258] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:14.245321 9417 tablet_replica.cc:330] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: stopping tablet replica
I20260810 11:49:14.246073 9417 raft_consensus.cc:2229] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:14.248818 9417 raft_consensus.cc:2258] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:14.249367 9419 ts_tablet_manager.cc:1905] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:14.249439 9418 tablet_replica.cc:330] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148: stopping tablet replica
I20260810 11:49:14.251792 9417 ts_tablet_manager.cc:1905] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:14.252323 9418 raft_consensus.cc:2229] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:14.252907 9418 raft_consensus.cc:2258] T 9b16c83895714d9bbf51241b31058a2f P 4fcf581e6c5c4297bdd23a046bc7c148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:14.254834 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.236805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.20.93.129:40813)
0810 11:49:14.237033 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.237042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.245680 (+ 8638us) server_negotiation.cc:407] Connection header received
0810 11:49:14.246043 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.246053 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.246212 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.246309 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:14.247178 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.248735 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.250663 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.251321 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.251419 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.252986 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:14.252997 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:14.253014 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:14.253862 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:14.253928 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:14.253957 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:14.254056 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.254317 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.254444 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:49:14.256031 9083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:14.258137 9419 ts_tablet_manager.cc:1918] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:14.258455 9419 log.cc:1196] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/9b16c83895714d9bbf51241b31058a2f
I20260810 11:49:14.259274 9419 ts_tablet_manager.cc:1939] T 9b16c83895714d9bbf51241b31058a2f P 7b35e541350c4cc6956181a498034829: Deleting consensus metadata
I20260810 11:49:14.261178 9417 ts_tablet_manager.cc:1918] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:14.261418 9417 log.cc:1196] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/9b16c83895714d9bbf51241b31058a2f
I20260810 11:49:14.261878 8868 catalog_manager.cc:4841] TS 7b35e541350c4cc6956181a498034829 (127.20.93.131:32991): tablet 9b16c83895714d9bbf51241b31058a2f (table test-table [id=109efd0b691e46e792fd412604274234]) successfully deleted
I20260810 11:49:14.262087 9417 ts_tablet_manager.cc:1939] T 9b16c83895714d9bbf51241b31058a2f P 85e53d26ef0d462fb74c86ee023ee237: Deleting consensus metadata
I20260810 11:49:14.263963 8868 catalog_manager.cc:4841] TS 85e53d26ef0d462fb74c86ee023ee237 (127.20.93.130:46491): tablet 9b16c83895714d9bbf51241b31058a2f (table test-table [id=109efd0b691e46e792fd412604274234]) successfully deleted
I20260810 11:49:14.364745 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8948
W20260810 11:49:14.390203 8871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.93.129:40813 (error 108)
I20260810 11:49:14.399273 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9092
I20260810 11:49:14.403223 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.402745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36302)
0810 11:49:14.402933 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.403073 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:49:14.405120 8871 catalog_manager.cc:4584] TS 4fcf581e6c5c4297bdd23a046bc7c148 (127.20.93.129:40813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b16c83895714d9bbf51241b31058a2f: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
I20260810 11:49:14.408787 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.400493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.20.93.130:46491)
0810 11:49:14.400675 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.400681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.400701 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:49:14.400767 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.400773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.400895 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.401008 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:14.401666 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.402911 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.404899 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.405641 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.405764 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.407010 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:14.407022 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:14.407040 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:14.407894 (+ 854us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:14.407954 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:14.407988 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:14.408093 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.408315 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.408434 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:49:14.409785 9225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:14.458846 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.458258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36304)
0810 11:49:14.458519 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.458654 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 11:49:14.460134 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.459689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36312)
0810 11:49:14.459863 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.459971 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 11:49:14.506301 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9092
I20260810 11:49:14.521052 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.520416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36318)
0810 11:49:14.520690 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.520820 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 11:49:14.522439 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.521948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36334)
0810 11:49:14.522155 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.522269 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 11:49:14.528028 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9233
I20260810 11:49:14.536883 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.528905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.20.93.131:32991)
0810 11:49:14.529144 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:14.529150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:14.529187 (+ 37us) server_negotiation.cc:407] Connection header received
0810 11:49:14.529265 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:14.529271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:14.529413 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:14.529543 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:14.530097 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.531401 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.532797 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:14.533509 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:14.533693 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:14.534930 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:14.534943 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:14.534971 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:14.535967 (+ 996us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:14.536041 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:14.536069 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:14.536212 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:14.536353 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 11:49:14.536468 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:49:14.537756 9368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:14.603510 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.602877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36348)
0810 11:49:14.603160 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.603279 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 11:49:14.604775 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.604301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36352)
0810 11:49:14.604537 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.604631 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 11:49:14.665484 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9233
I20260810 11:49:14.688342 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 8853
I20260810 11:49:14.689433 8916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:14.800275 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:14.799412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:40813 (local address 127.0.0.1:36368)
0810 11:49:14.799697 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:14.799918 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 11:49:14.800479 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 8853
2026-08-10T11:49:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3041 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:14 dist-test-slave-njxd krb5kdc[9429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:14 dist-test-slave-njxd krb5kdc[9429](info): set up 2 sockets
Aug 10 11:49:14 dist-test-slave-njxd krb5kdc[9429](info): commencing operation
krb5kdc: starting...
W20260810 11:49:15.988518 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:16 dist-test-slave-njxd krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:16.321713 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44247
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:16.675097 9442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:16.675405 9442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:16.675583 9442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:16.683214 9442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:16.683341 9442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:16.683445 9442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:16.683550 9442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:16.683636 9442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:16.693938 9442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:16.695360 9442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:16.696720 9442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:16.710009 9450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:16.709960 9451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:16.709923 9453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:16.715260 9442 server_base.cc:1034] running on GCE node
I20260810 11:49:16.716200 9442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:16.717839 9442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:16.719082 9442 hybrid_clock.cc:648] HybridClock initialized: now 1786362556719027 us; error 51 us; skew 500 ppm
I20260810 11:49:16.719532 9442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:16 dist-test-slave-njxd krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:16.723019 9442 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:16.724758 9442 webserver.cc:460] Webserver started at http://127.20.93.190:41461/ using document root <none> and password file <none>
I20260810 11:49:16.725562 9442 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:16.725751 9442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:16.726034 9442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:16.729096 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "2f2a27a423344cc4bc4225e11da9ff72"
format_stamp: "Formatted at 2026-08-10 11:49:16 on dist-test-slave-njxd"
server_key: "c1923ad4aa92be20872001e33bad66e7"
server_key_iv: "94d18cee8bc78ac3c334e399c4da0cd3"
server_key_version: "encryptionkey@0"
I20260810 11:49:16.729913 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "2f2a27a423344cc4bc4225e11da9ff72"
format_stamp: "Formatted at 2026-08-10 11:49:16 on dist-test-slave-njxd"
server_key: "c1923ad4aa92be20872001e33bad66e7"
server_key_iv: "94d18cee8bc78ac3c334e399c4da0cd3"
server_key_version: "encryptionkey@0"
I20260810 11:49:16.735316 9442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:49:16.738968 9462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:16.740406 9442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:49:16.740582 9442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "2f2a27a423344cc4bc4225e11da9ff72"
format_stamp: "Formatted at 2026-08-10 11:49:16 on dist-test-slave-njxd"
server_key: "c1923ad4aa92be20872001e33bad66e7"
server_key_iv: "94d18cee8bc78ac3c334e399c4da0cd3"
server_key_version: "encryptionkey@0"
I20260810 11:49:16.740756 9442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:16.774588 9442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:16.778754 9442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:16.778998 9442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:16.798362 9442 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44247
I20260810 11:49:16.798379 9518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44247 every 8 connection(s)
I20260810 11:49:16.799845 9442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:16.803774 9519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:16.804399 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9442
I20260810 11:49:16.804610 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:16.805203 20854 external_mini_cluster.cc:1428] Setting key ebb810fe80b8940aad0a2bc911874ccd
I20260810 11:49:16.818507 9519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: Bootstrap starting.
Aug 10 11:49:16 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:16.822650 9519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:16.824147 9519 log.cc:824] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:16.827782 9519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: No bootstrap required, opened a new log
I20260810 11:49:16.831775 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:16.807151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.20.93.190:44247)
0810 11:49:16.808022 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:16.808044 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:16.808087 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:49:16.810374 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:16.810400 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:16.810827 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:16.811197 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:16.813076 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:16.814489 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:16.816465 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:16.817155 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:16.817290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:16.822555 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:16.822570 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:16.822593 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:16.822660 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:16.826712 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:16.827507 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:16.827515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:16.827523 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:16.827743 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:16.828216 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:16.828224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:16.828227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:16.829018 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:16.829324 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:16.829767 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 11:49:16.830288 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":439,"threads_started":1}
I20260810 11:49:16.836979 9519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } }
I20260810 11:49:16.837352 9519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:16.837430 9519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2a27a423344cc4bc4225e11da9ff72, State: Initialized, Role: FOLLOWER
I20260810 11:49:16.838030 9519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } }
I20260810 11:49:16.838245 9519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:16.838347 9519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:16.838480 9519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:16.840678 9519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } }
I20260810 11:49:16.841251 9519 leader_election.cc:302] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2a27a423344cc4bc4225e11da9ff72; no voters:
I20260810 11:49:16.841941 9519 leader_election.cc:288] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:16.842078 9524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:16.842823 9524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2a27a423344cc4bc4225e11da9ff72, State: Running, Role: LEADER
I20260810 11:49:16.843335 9524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } }
I20260810 11:49:16.844183 9519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:16.846486 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f2a27a423344cc4bc4225e11da9ff72. Latest consensus state: current_term: 1 leader_uuid: "2f2a27a423344cc4bc4225e11da9ff72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } } }
I20260810 11:49:16.846448 9525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f2a27a423344cc4bc4225e11da9ff72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2a27a423344cc4bc4225e11da9ff72" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44247 } } }
I20260810 11:49:16.847522 9525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:16.848841 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:16.849239 9531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:16.856200 9531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:16.868299 9531 catalog_manager.cc:1347] Generated new cluster ID: 97b35e61c62a4e9f8090441a2e5a8a0e
I20260810 11:49:16.868436 9531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:16.888605 9531 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:16.890560 9531 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:16.899192 9531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: Generated new TSK 0
I20260810 11:49:16.900053 9531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.93.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
Entry for principal kudu/127.20.93.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab.
I20260810 11:49:16.998771 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:17.375334 9547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:17.375635 9547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:17.375734 9547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:17.383129 9547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:17.383241 9547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:17.383354 9547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:17.393322 9547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.129.keytab
--principal=kudu/127.20.93.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:17.394691 9547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:17.395965 9547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:17.409689 9553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:17.409718 9557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:17.409631 9555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:17.410523 9547 server_base.cc:1034] running on GCE node
I20260810 11:49:17.411123 9547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:17.412089 9547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:17.413332 9547 hybrid_clock.cc:648] HybridClock initialized: now 1786362557413282 us; error 66 us; skew 500 ppm
I20260810 11:49:17.413769 9547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:17 dist-test-slave-njxd krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362557, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:17.417233 9547 init.cc:377] Logged in from keytab as kudu/127.20.93.129@KRBTEST.COM (short username kudu)
I20260810 11:49:17.418846 9547 webserver.cc:460] Webserver started at http://127.20.93.129:39473/ using document root <none> and password file <none>
I20260810 11:49:17.419531 9547 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:17.419644 9547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:17.419914 9547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:17.423071 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "6d6c926405ab46d986821f236892dd8b"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "f88849461b431c5562927a449d3098e4"
server_key_iv: "7fe1305deb9ee523cad6e5badc88b2a9"
server_key_version: "encryptionkey@0"
I20260810 11:49:17.423869 9547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "6d6c926405ab46d986821f236892dd8b"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "f88849461b431c5562927a449d3098e4"
server_key_iv: "7fe1305deb9ee523cad6e5badc88b2a9"
server_key_version: "encryptionkey@0"
I20260810 11:49:17.429245 9547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:49:17.433122 9565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:17.434882 9547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:17.435056 9547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "6d6c926405ab46d986821f236892dd8b"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "f88849461b431c5562927a449d3098e4"
server_key_iv: "7fe1305deb9ee523cad6e5badc88b2a9"
server_key_version: "encryptionkey@0"
I20260810 11:49:17.435262 9547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:17.456475 9547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:17.461627 9547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:17.461906 9547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:17.462962 9547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:17.464536 9547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:17.464613 9547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:17.464705 9547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:17.464749 9547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:17.501305 9547 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:36899
I20260810 11:49:17.501359 9684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:36899 every 8 connection(s)
I20260810 11:49:17.502766 9547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:17.505131 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9547
I20260810 11:49:17.505342 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:17.505937 20854 external_mini_cluster.cc:1428] Setting key d2a2636c3169367f48b8506eb71ab2ce
Aug 10 11:49:17 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362557, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.129@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:17.525061 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:17.505683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:58949 (local address 127.20.93.190:44247)
0810 11:49:17.506223 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:17.506227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:17.507521 (+ 1294us) server_negotiation.cc:407] Connection header received
0810 11:49:17.509723 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:17.509728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:17.509826 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:17.509955 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:17.511618 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:17.512364 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:17.514083 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:17.514526 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:17.514678 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:17.518486 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:17.518491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:17.518495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:17.518532 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:17.521670 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:17.522514 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:17.522522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:17.522530 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:17.522671 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:17.523143 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:17.523147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:17.523150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:17.523437 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:17.523603 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:17.524405 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 11:49:17.524665 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"spinlock_wait_cycles":9600,"thread_start_us":348,"threads_started":1}
I20260810 11:49:17.526240 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:17.506398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44247 (local address 127.20.93.129:58949)
0810 11:49:17.507321 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:17.507383 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:17.509384 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:17.510228 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:17.510238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:17.510806 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:17.511431 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:17.511449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:17.512544 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:17.512550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:17.513924 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:17.513936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:17.514781 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:17.514789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:17.515053 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:17.515799 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:17.515828 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:17.518212 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:17.521844 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:17.521852 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:17.521867 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:17.522354 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:17.522794 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:17.522799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:17.522804 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:17.523003 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:17.523598 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:17.523606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:17.523984 (+ 378us) client_negotiation.cc:769] Sending connection context
0810 11:49:17.524277 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:49:17.524915 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":350,"threads_started":1}
I20260810 11:49:17.528198 9685 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44247
I20260810 11:49:17.528577 9685 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:17.529443 9685 heartbeater.cc:511] Master 127.20.93.190:44247 requested a full tablet report, sending...
I20260810 11:49:17.531883 9482 ts_manager.cc:194] Registered new tserver with Master: 6d6c926405ab46d986821f236892dd8b (127.20.93.129:36899)
I20260810 11:49:17.533738 9482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.129@KRBTEST.COM'} at 127.20.93.129:58949
WARNING: no policy specified for kudu/127.20.93.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
Entry for principal kudu/127.20.93.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab.
I20260810 11:49:17.581450 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:17.942276 9692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:17.942606 9692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:17.942780 9692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:17.950984 9692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:17.951123 9692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:17.951306 9692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:17.962277 9692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.130.keytab
--principal=kudu/127.20.93.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:17.963795 9692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:17.965333 9692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:17.979841 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:17.979952 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:17.979951 9701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:17.980867 9692 server_base.cc:1034] running on GCE node
I20260810 11:49:17.981530 9692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:17.982659 9692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:17.983870 9692 hybrid_clock.cc:648] HybridClock initialized: now 1786362557983816 us; error 45 us; skew 500 ppm
I20260810 11:49:17.984202 9692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:17 dist-test-slave-njxd krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362557, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:17.987624 9692 init.cc:377] Logged in from keytab as kudu/127.20.93.130@KRBTEST.COM (short username kudu)
I20260810 11:49:17.989332 9692 webserver.cc:460] Webserver started at http://127.20.93.130:33235/ using document root <none> and password file <none>
I20260810 11:49:17.990139 9692 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:17.990257 9692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:17.990540 9692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:17.993674 9692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "cefc508f9a544c2ab98871f16118d71f"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "ec6b4de78dca70be38cdd2d0c1c4823b"
server_key_iv: "141993b96cb4c73d94d74df896687d49"
server_key_version: "encryptionkey@0"
I20260810 11:49:17.994468 9692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "cefc508f9a544c2ab98871f16118d71f"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "ec6b4de78dca70be38cdd2d0c1c4823b"
server_key_iv: "141993b96cb4c73d94d74df896687d49"
server_key_version: "encryptionkey@0"
I20260810 11:49:18.000241 9692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:18.004017 9708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.005671 9692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:49:18.005822 9692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "cefc508f9a544c2ab98871f16118d71f"
format_stamp: "Formatted at 2026-08-10 11:49:17 on dist-test-slave-njxd"
server_key: "ec6b4de78dca70be38cdd2d0c1c4823b"
server_key_iv: "141993b96cb4c73d94d74df896687d49"
server_key_version: "encryptionkey@0"
I20260810 11:49:18.006008 9692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:18.027364 9692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:18.031803 9692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:18.032073 9692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:18.033004 9692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:18.034428 9692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:18.034507 9692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.034607 9692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:18.034655 9692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.071102 9692 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:34151
I20260810 11:49:18.071241 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:34151 every 8 connection(s)
I20260810 11:49:18.072510 9692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:18.076828 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9692
I20260810 11:49:18.077020 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:18.077500 20854 external_mini_cluster.cc:1428] Setting key c64167cda7e05a9412e7f8faebeea811
Aug 10 11:49:18 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362557, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.130@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:18.095912 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.075213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:57375 (local address 127.20.93.190:44247)
0810 11:49:18.075702 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.075710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.076689 (+ 979us) server_negotiation.cc:407] Connection header received
0810 11:49:18.078895 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.078901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.078995 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.079116 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:18.081218 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.082057 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.084568 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.085031 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.085130 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.089686 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:18.089692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:18.089695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:18.089735 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:18.092530 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.093384 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.093392 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.093400 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.093537 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.094052 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.094057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.094061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.094359 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:18.094526 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.095278 (+ 752us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.095531 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":223,"threads_started":1}
I20260810 11:49:18.096966 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.075821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44247 (local address 127.20.93.130:57375)
0810 11:49:18.076521 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.076568 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.078629 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.079366 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.079375 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.080142 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:18.080991 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.081026 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.082221 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.082230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.084414 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.084432 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.085268 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.085278 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.085631 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.086413 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:18.086452 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:18.089374 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:18.092724 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:18.092734 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:18.092753 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:18.093237 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:18.093709 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:18.093715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:18.093718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:18.093926 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:18.094504 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:18.094525 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:18.094902 (+ 377us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.095185 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.095768 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":266,"threads_started":1}
I20260810 11:49:18.098634 9828 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44247
I20260810 11:49:18.098985 9828 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:18.099957 9828 heartbeater.cc:511] Master 127.20.93.190:44247 requested a full tablet report, sending...
I20260810 11:49:18.101692 9482 ts_manager.cc:194] Registered new tserver with Master: cefc508f9a544c2ab98871f16118d71f (127.20.93.130:34151)
I20260810 11:49:18.102560 9482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.130@KRBTEST.COM'} at 127.20.93.130:57375
WARNING: no policy specified for kudu/127.20.93.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
Entry for principal kudu/127.20.93.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab.
I20260810 11:49:18.153368 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:18.499190 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:18.499485 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:18.499665 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:18.510154 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:18.510281 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:18.510481 9834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:18.520922 9834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.131.keytab
--principal=kudu/127.20.93.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:18.522408 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:18.523964 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:49:18.537454 9685 heartbeater.cc:503] Master 127.20.93.190:44247 was elected leader, sending a full tablet report...
W20260810 11:49:18.538239 9841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:18.538461 9843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:18.538798 9840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:18.539425 9834 server_base.cc:1034] running on GCE node
I20260810 11:49:18.540194 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:18.541297 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:18.542502 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786362558542468 us; error 56 us; skew 500 ppm
I20260810 11:49:18.542861 9834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:18 dist-test-slave-njxd krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362558, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:18.546553 9834 init.cc:377] Logged in from keytab as kudu/127.20.93.131@KRBTEST.COM (short username kudu)
I20260810 11:49:18.548187 9834 webserver.cc:460] Webserver started at http://127.20.93.131:43557/ using document root <none> and password file <none>
I20260810 11:49:18.548910 9834 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:18.549015 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:18.549301 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:18.552579 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "e076c288c2ba4516af8023b0a072cb26"
format_stamp: "Formatted at 2026-08-10 11:49:18 on dist-test-slave-njxd"
server_key: "2eae12e1c13410f9f6e1b4f99247b85c"
server_key_iv: "d1e1af9c84e21347581a3ef07e754352"
server_key_version: "encryptionkey@0"
I20260810 11:49:18.553398 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "e076c288c2ba4516af8023b0a072cb26"
format_stamp: "Formatted at 2026-08-10 11:49:18 on dist-test-slave-njxd"
server_key: "2eae12e1c13410f9f6e1b4f99247b85c"
server_key_iv: "d1e1af9c84e21347581a3ef07e754352"
server_key_version: "encryptionkey@0"
I20260810 11:49:18.559497 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260810 11:49:18.563406 9852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.564913 9834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:49:18.565111 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "e076c288c2ba4516af8023b0a072cb26"
format_stamp: "Formatted at 2026-08-10 11:49:18 on dist-test-slave-njxd"
server_key: "2eae12e1c13410f9f6e1b4f99247b85c"
server_key_iv: "d1e1af9c84e21347581a3ef07e754352"
server_key_version: "encryptionkey@0"
I20260810 11:49:18.565332 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:18.597636 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:18.602078 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:18.602389 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:18.603389 9834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:18.604770 9834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:18.604877 9834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.605026 9834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:18.605129 9834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:18.639812 9834 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:34345
I20260810 11:49:18.639868 9971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:34345 every 8 connection(s)
I20260810 11:49:18.641333 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:18.646764 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 9834
I20260810 11:49:18.646962 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:18.647439 20854 external_mini_cluster.cc:1428] Setting key 048438cbeb1e3ad3dccb9ed3b86d9276
Aug 10 11:49:18 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362558, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.131@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:18.663131 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.644196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:54971 (local address 127.20.93.190:44247)
0810 11:49:18.644735 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.644740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.645924 (+ 1184us) server_negotiation.cc:407] Connection header received
0810 11:49:18.648114 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.648122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.648270 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.648399 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:18.650638 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.651456 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.653006 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.653416 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.653514 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.656990 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:18.656995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:18.656998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:18.657032 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:18.659810 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.660514 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.660518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.660520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.660643 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.661043 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.661047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.661050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.661412 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:18.661598 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.662522 (+ 924us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.662754 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":274,"threads_started":1}
I20260810 11:49:18.664199 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.644887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44247 (local address 127.20.93.131:54971)
0810 11:49:18.645765 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.645805 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.647739 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.648712 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.648732 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.649485 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:18.650432 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.650460 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.651597 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.651601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.652866 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.652878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.653663 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.653669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.653884 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.654563 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:49:18.654585 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:18.656784 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:18.659944 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:18.659951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:18.659965 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:18.660402 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:18.660734 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:49:18.660737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:49:18.660739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:49:18.660918 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:49:18.661540 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:18.661545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:18.662088 (+ 543us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.662394 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.662944 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":295,"threads_started":1}
I20260810 11:49:18.665548 9972 heartbeater.cc:347] Connected to a master server at 127.20.93.190:44247
I20260810 11:49:18.665908 9972 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:18.666669 9972 heartbeater.cc:511] Master 127.20.93.190:44247 requested a full tablet report, sending...
I20260810 11:49:18.668192 9482 ts_manager.cc:194] Registered new tserver with Master: e076c288c2ba4516af8023b0a072cb26 (127.20.93.131:34345)
I20260810 11:49:18.669054 9482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.93.131@KRBTEST.COM'} at 127.20.93.131:54971
I20260810 11:49:18.675935 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:18.692153 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.680074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.20.93.190:44247)
0810 11:49:18.680320 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.680329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.680681 (+ 352us) server_negotiation.cc:407] Connection header received
0810 11:49:18.680822 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.680825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.680902 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.680985 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:18.682557 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.683319 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.684547 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.684926 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.685015 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.687328 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:18.687332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:18.687335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:18.687363 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:18.689777 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.690422 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.690425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.690427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.690547 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:18.690865 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:18.690868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:18.690870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:18.691166 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:18.691358 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.691646 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.691810 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:49:18.695863 9482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:18.702672 9482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:18.739172 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.716092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.20.93.130:34151)
0810 11:49:18.729869 (+ 13777us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.729882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.729924 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:49:18.730068 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.730080 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.730472 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.730738 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:18.731854 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.733400 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.736250 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.737876 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.738018 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.738245 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.738650 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.738838 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13545,"thread_start_us":13358,"threads_started":1}
I20260810 11:49:18.739235 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.715939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:34151 (local address 127.0.0.1:50142)
0810 11:49:18.727277 (+ 11338us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.727335 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.727553 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.731102 (+ 3549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.731111 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.731145 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:18.731640 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.731654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.733547 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.733552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.736031 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.736046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.738114 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.738120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.738348 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.738414 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.738582 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.738782 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11050,"thread_start_us":16798,"threads_started":1}
I20260810 11:49:18.740906 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.716623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.20.93.129:36899)
0810 11:49:18.731253 (+ 14630us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.731266 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.731302 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:49:18.732419 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.732438 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.732707 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.732924 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:18.733800 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.735270 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.737450 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.738874 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.739013 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.740456 (+ 1443us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.740540 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.740679 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14402,"thread_start_us":14225,"threads_started":1}
I20260810 11:49:18.741250 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.716000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:36899 (local address 127.0.0.1:45718)
0810 11:49:18.730297 (+ 14297us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.730332 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.730510 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.733001 (+ 2491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.733007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.733041 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:18.733547 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.733558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.735451 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.735457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.737279 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.737294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.739270 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.739279 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.739707 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.739747 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.739839 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.739965 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14106,"thread_start_us":10836,"threads_started":1}
I20260810 11:49:18.742281 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.717342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:34345 (local address 127.0.0.1:42676)
0810 11:49:18.733029 (+ 15687us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.733053 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.733276 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.734000 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.734004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.734029 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:18.734470 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.734483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.736450 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.736455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.738591 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.738607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.741435 (+ 2828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.741442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.741669 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.741706 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.741801 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.741925 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15567,"mutex_wait_us":22,"thread_start_us":13746,"threads_started":1}
I20260810 11:49:18.742565 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.717784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.20.93.131:34345)
0810 11:49:18.724867 (+ 7083us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.724881 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.733165 (+ 8284us) server_negotiation.cc:407] Connection header received
0810 11:49:18.733492 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.733499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.733789 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.734003 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:18.734690 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.736305 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.738782 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.740936 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.741061 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.741286 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.741868 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.742022 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6857,"thread_start_us":6677,"threads_started":1}
I20260810 11:49:18.743726 9758 tablet_service.cc:1467] Processing CreateTablet for tablet e4b0a1fce8bd472d87b840430a29f13e (DEFAULT_TABLE table=test-table [id=5e9b69a702ac4accbd861f77e23a7563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:18.745157 9758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b0a1fce8bd472d87b840430a29f13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:18.744758 9614 tablet_service.cc:1467] Processing CreateTablet for tablet e4b0a1fce8bd472d87b840430a29f13e (DEFAULT_TABLE table=test-table [id=5e9b69a702ac4accbd861f77e23a7563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:18.746193 9614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b0a1fce8bd472d87b840430a29f13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:18.746304 9901 tablet_service.cc:1467] Processing CreateTablet for tablet e4b0a1fce8bd472d87b840430a29f13e (DEFAULT_TABLE table=test-table [id=5e9b69a702ac4accbd861f77e23a7563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:18.747689 9901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b0a1fce8bd472d87b840430a29f13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:18.756723 9991 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Bootstrap starting.
I20260810 11:49:18.760584 9991 tablet_bootstrap.cc:654] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:18.760818 9992 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Bootstrap starting.
I20260810 11:49:18.762138 9991 log.cc:824] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:18.762405 9993 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Bootstrap starting.
I20260810 11:49:18.764353 9992 tablet_bootstrap.cc:654] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:18.765450 9993 tablet_bootstrap.cc:654] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:18.765653 9992 log.cc:824] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:18.766004 9991 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: No bootstrap required, opened a new log
I20260810 11:49:18.766332 9991 ts_tablet_manager.cc:1397] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 11:49:18.766736 9993 log.cc:824] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:18.768764 9992 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: No bootstrap required, opened a new log
I20260810 11:49:18.769071 9992 ts_tablet_manager.cc:1397] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 11:49:18.770792 9993 tablet_bootstrap.cc:492] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: No bootstrap required, opened a new log
I20260810 11:49:18.771085 9993 ts_tablet_manager.cc:1397] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:49:18.776921 9992 raft_consensus.cc:348] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.777381 9992 raft_consensus.cc:374] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:18.777478 9992 raft_consensus.cc:729] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6c926405ab46d986821f236892dd8b, State: Initialized, Role: FOLLOWER
I20260810 11:49:18.778153 9992 consensus_queue.cc:260] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.778867 9991 raft_consensus.cc:348] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.779362 9991 raft_consensus.cc:374] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:18.779464 9991 raft_consensus.cc:729] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefc508f9a544c2ab98871f16118d71f, State: Initialized, Role: FOLLOWER
I20260810 11:49:18.780853 9992 ts_tablet_manager.cc:1428] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260810 11:49:18.781278 9991 consensus_queue.cc:260] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.782539 9828 heartbeater.cc:503] Master 127.20.93.190:44247 was elected leader, sending a full tablet report...
I20260810 11:49:18.783195 9991 ts_tablet_manager.cc:1428] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260810 11:49:18.783072 9993 raft_consensus.cc:348] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.783556 9993 raft_consensus.cc:374] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:18.783681 9993 raft_consensus.cc:729] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e076c288c2ba4516af8023b0a072cb26, State: Initialized, Role: FOLLOWER
I20260810 11:49:18.784368 9993 consensus_queue.cc:260] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.785836 9972 heartbeater.cc:503] Master 127.20.93.190:44247 was elected leader, sending a full tablet report...
I20260810 11:49:18.786386 9993 ts_tablet_manager.cc:1428] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260810 11:49:18.824644 9829 tablet.cc:2369] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:18.857751 9998 raft_consensus.cc:482] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:18.858139 9998 raft_consensus.cc:504] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.859915 9998 leader_election.cc:288] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cefc508f9a544c2ab98871f16118d71f (127.20.93.130:34151), e076c288c2ba4516af8023b0a072cb26 (127.20.93.131:34345)
I20260810 11:49:18.866763 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.860329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:53971 (local address 127.20.93.131:34345)
0810 11:49:18.860560 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.860568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.861453 (+ 885us) server_negotiation.cc:407] Connection header received
0810 11:49:18.861669 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.861676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.861741 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.861835 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:18.862514 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.863210 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.865019 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.865842 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.866012 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.866095 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.866381 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.866505 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:49:18.866868 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.860300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:54313 (local address 127.20.93.130:34151)
0810 11:49:18.860615 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.860622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.861360 (+ 738us) server_negotiation.cc:407] Connection header received
0810 11:49:18.861646 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.861652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.861726 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.861829 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:18.862450 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.863166 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.865024 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.865909 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.866095 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.866178 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.866467 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.866611 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260810 11:49:18.867134 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.860503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:34345 (local address 127.20.93.129:53971)
0810 11:49:18.861340 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.861354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.861502 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.861861 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.861865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.861889 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:18.862325 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.862334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.863342 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.863348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.864860 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.864871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.866059 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.866066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.866217 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.866244 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.866352 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.866474 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":4,"thread_start_us":492,"threads_started":1}
I20260810 11:49:18.867304 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.860386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:34151 (local address 127.20.93.129:54313)
0810 11:49:18.861242 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:18.861263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:18.861441 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:18.861846 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:18.861851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:18.861874 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:18.862299 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.862311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.863299 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.863307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.864862 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:18.864874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.866149 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.866155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:18.866313 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.866336 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:49:18.866441 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 11:49:18.866560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":39,"thread_start_us":493,"threads_started":1}
I20260810 11:49:18.868278 9923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b0a1fce8bd472d87b840430a29f13e" candidate_uuid: "6d6c926405ab46d986821f236892dd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e076c288c2ba4516af8023b0a072cb26" is_pre_election: true
I20260810 11:49:18.868278 9780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b0a1fce8bd472d87b840430a29f13e" candidate_uuid: "6d6c926405ab46d986821f236892dd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cefc508f9a544c2ab98871f16118d71f" is_pre_election: true
I20260810 11:49:18.868873 9780 raft_consensus.cc:2454] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d6c926405ab46d986821f236892dd8b in term 0.
I20260810 11:49:18.868872 9923 raft_consensus.cc:2454] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d6c926405ab46d986821f236892dd8b in term 0.
I20260810 11:49:18.869616 9569 leader_election.cc:302] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d6c926405ab46d986821f236892dd8b, cefc508f9a544c2ab98871f16118d71f; no voters:
I20260810 11:49:18.870028 9998 raft_consensus.cc:2781] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:18.870206 9998 raft_consensus.cc:482] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:18.870357 9998 raft_consensus.cc:3037] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:18.872635 9998 raft_consensus.cc:504] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.873239 9998 leader_election.cc:288] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [CANDIDATE]: Term 1 election: Requested vote from peers cefc508f9a544c2ab98871f16118d71f (127.20.93.130:34151), e076c288c2ba4516af8023b0a072cb26 (127.20.93.131:34345)
I20260810 11:49:18.873766 9923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b0a1fce8bd472d87b840430a29f13e" candidate_uuid: "6d6c926405ab46d986821f236892dd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e076c288c2ba4516af8023b0a072cb26"
I20260810 11:49:18.873770 9780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b0a1fce8bd472d87b840430a29f13e" candidate_uuid: "6d6c926405ab46d986821f236892dd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cefc508f9a544c2ab98871f16118d71f"
I20260810 11:49:18.873974 9923 raft_consensus.cc:3037] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:18.873991 9780 raft_consensus.cc:3037] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:18.876256 9923 raft_consensus.cc:2454] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d6c926405ab46d986821f236892dd8b in term 1.
I20260810 11:49:18.876332 9780 raft_consensus.cc:2454] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d6c926405ab46d986821f236892dd8b in term 1.
I20260810 11:49:18.876775 9568 leader_election.cc:302] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d6c926405ab46d986821f236892dd8b, e076c288c2ba4516af8023b0a072cb26; no voters:
I20260810 11:49:18.877143 9998 raft_consensus.cc:2781] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:18.877712 9998 raft_consensus.cc:686] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 1 LEADER]: Becoming Leader. State: Replica: 6d6c926405ab46d986821f236892dd8b, State: Running, Role: LEADER
I20260810 11:49:18.878217 9998 consensus_queue.cc:237] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } }
I20260810 11:49:18.883121 9482 catalog_manager.cc:5495] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d6c926405ab46d986821f236892dd8b (127.20.93.129). New cstate: current_term: 1 leader_uuid: "6d6c926405ab46d986821f236892dd8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d6c926405ab46d986821f236892dd8b" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 36899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 } health_report { overall_health: UNKNOWN } } }
W20260810 11:49:18.893106 9973 tablet.cc:2369] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:18.931820 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.926355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.20.93.129:36899)
0810 11:49:18.926614 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.926624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:18.926645 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:49:18.926748 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:18.926751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:18.926848 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:18.927010 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:18.927419 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.928171 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.929528 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:18.930011 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:18.930151 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:18.930681 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:18.931217 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:18.931386 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:18.931445 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 11:49:18.931530 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:49:18.940649 9923 raft_consensus.cc:1261] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 1 FOLLOWER]: Refusing update from remote peer 6d6c926405ab46d986821f236892dd8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:18.940689 9780 raft_consensus.cc:1261] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 1 FOLLOWER]: Refusing update from remote peer 6d6c926405ab46d986821f236892dd8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:18.941684 10003 consensus_queue.cc:1035] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cefc508f9a544c2ab98871f16118d71f" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 34151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:18.942080 9998 consensus_queue.cc:1035] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e076c288c2ba4516af8023b0a072cb26" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:18.974556 9482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:18.974838 9482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:18.978602 9482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f2a27a423344cc4bc4225e11da9ff72: Sending DeleteTablet for 3 replicas of tablet e4b0a1fce8bd472d87b840430a29f13e
I20260810 11:49:18.979843 9758 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b0a1fce8bd472d87b840430a29f13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:18 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:50142
I20260810 11:49:18.979887 9614 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b0a1fce8bd472d87b840430a29f13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:18 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:45718
I20260810 11:49:18.980497 9901 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b0a1fce8bd472d87b840430a29f13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:18 UTC) from {username='kudu', principal='kudu/127.20.93.190@KRBTEST.COM'} at 127.0.0.1:42676
I20260810 11:49:18.981895 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9547
I20260810 11:49:19.006270 10016 tablet_replica.cc:330] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: stopping tablet replica
I20260810 11:49:19.007164 10016 raft_consensus.cc:2229] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:19.008071 10016 raft_consensus.cc:2258] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:19.010622 10016 ts_tablet_manager.cc:1905] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:19.018677 10017 tablet_replica.cc:330] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: stopping tablet replica
I20260810 11:49:19.018916 10016 ts_tablet_manager.cc:1918] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:19.019126 10018 tablet_replica.cc:330] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: stopping tablet replica
I20260810 11:49:19.019164 10016 log.cc:1196] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/e4b0a1fce8bd472d87b840430a29f13e
I20260810 11:49:19.019788 10016 ts_tablet_manager.cc:1939] T e4b0a1fce8bd472d87b840430a29f13e P 6d6c926405ab46d986821f236892dd8b: Deleting consensus metadata
I20260810 11:49:19.019762 10018 raft_consensus.cc:2229] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:19.020365 10018 raft_consensus.cc:2258] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:19.020933 10017 raft_consensus.cc:2229] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:19.021512 10017 raft_consensus.cc:2258] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 11:49:19 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.129@KRBTEST.COM
I20260810 11:49:19.021991 9464 catalog_manager.cc:4841] TS 6d6c926405ab46d986821f236892dd8b (127.20.93.129:36899): tablet e4b0a1fce8bd472d87b840430a29f13e (table test-table [id=5e9b69a702ac4accbd861f77e23a7563]) successfully deleted
I20260810 11:49:19.025424 10017 ts_tablet_manager.cc:1905] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:19.028841 10018 ts_tablet_manager.cc:1905] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:19.031523 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:18.983605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.20.93.129:36899)
0810 11:49:18.984113 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:18.984119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:19.002104 (+ 17985us) server_negotiation.cc:407] Connection header received
0810 11:49:19.002832 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:19.002841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:19.003066 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:19.003175 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:19.006840 (+ 3665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.008472 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.014305 (+ 5833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.015424 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.015577 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:19.023178 (+ 7601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:19.023192 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:19.023216 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:19.023276 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:19.027464 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.028504 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.028513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.028517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.028731 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.029292 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.029297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.029301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.029834 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:19.030201 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:19.030859 (+ 658us) server_negotiation.cc:299] Negotiation successful
0810 11:49:19.031128 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381}
I20260810 11:49:19.032866 10017 ts_tablet_manager.cc:1918] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:19.033056 10017 log.cc:1196] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/e4b0a1fce8bd472d87b840430a29f13e
I20260810 11:49:19.033159 9679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:19.033739 10017 ts_tablet_manager.cc:1939] T e4b0a1fce8bd472d87b840430a29f13e P cefc508f9a544c2ab98871f16118d71f: Deleting consensus metadata
I20260810 11:49:19.036037 9466 catalog_manager.cc:4841] TS cefc508f9a544c2ab98871f16118d71f (127.20.93.130:34151): tablet e4b0a1fce8bd472d87b840430a29f13e (table test-table [id=5e9b69a702ac4accbd861f77e23a7563]) successfully deleted
I20260810 11:49:19.038038 10018 ts_tablet_manager.cc:1918] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:19.038223 10018 log.cc:1196] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/e4b0a1fce8bd472d87b840430a29f13e
I20260810 11:49:19.038707 10018 ts_tablet_manager.cc:1939] T e4b0a1fce8bd472d87b840430a29f13e P e076c288c2ba4516af8023b0a072cb26: Deleting consensus metadata
I20260810 11:49:19.040373 9465 catalog_manager.cc:4841] TS e076c288c2ba4516af8023b0a072cb26 (127.20.93.131:34345): tablet e4b0a1fce8bd472d87b840430a29f13e (table test-table [id=5e9b69a702ac4accbd861f77e23a7563]) successfully deleted
I20260810 11:49:19.141943 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9547
I20260810 11:49:19.164342 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9692
Aug 10 11:49:19 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.130@KRBTEST.COM
I20260810 11:49:19.180645 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:19.165327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.20.93.130:34151)
0810 11:49:19.165527 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:19.165534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:19.165611 (+ 77us) server_negotiation.cc:407] Connection header received
0810 11:49:19.165688 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:19.165693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:19.165776 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:19.165888 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:19.167247 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.168072 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.169437 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.169936 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.170084 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:19.174496 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:19.174523 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:19.174540 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:19.174583 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:19.177755 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.178451 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.178458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.178465 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.178595 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.178993 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.178998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.179004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.179407 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:19.179722 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:19.180061 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 11:49:19.180287 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:49:19.181417 9822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:19.275772 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9692
I20260810 11:49:19.298301 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9834
Aug 10 11:49:19 dist-test-slave-njxd krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.131@KRBTEST.COM
I20260810 11:49:19.314693 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:19.299191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.20.93.131:34345)
0810 11:49:19.299377 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:19.299382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:19.299401 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:49:19.299534 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:19.299538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:19.299638 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:19.299743 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:19.301083 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.301943 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.303274 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:19.303671 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:19.303781 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:19.308724 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:19.308741 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:19.308758 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:19.308798 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:19.311875 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.312669 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.312676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.312679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.312816 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:19.313194 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:19.313198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:19.313201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:19.313711 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:19.313960 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:19.314220 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 11:49:19.314393 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:49:19.315354 9966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:19.410637 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9834
I20260810 11:49:19.432845 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 9442
I20260810 11:49:19.434190 9514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:19.549364 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 9442
2026-08-10T11:49:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4752 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T11:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:19Z Disabled control of system clock
I20260810 11:49:19.648762 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:36285
--webserver_interface=127.20.93.190
--webserver_port=0
--builtin_ntp_servers=127.20.93.148:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:36285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:20.005738 10028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:20.006043 10028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:20.006184 10028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:20.013656 10028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:20.013761 10028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:20.013834 10028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:20.013934 10028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:20.014001 10028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:20.023386 10028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:36285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:20.024771 10028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:20.026790 10028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:20.039039 10033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:20.039086 10034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:20.039162 10036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:20.040155 10028 server_base.cc:1034] running on GCE node
I20260810 11:49:20.040992 10028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:20.042505 10028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:20.043751 10028 hybrid_clock.cc:648] HybridClock initialized: now 1786362560043708 us; error 38 us; skew 500 ppm
I20260810 11:49:20.044121 10028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:20.051716 10028 webserver.cc:460] Webserver started at http://127.20.93.190:45761/ using document root <none> and password file <none>
I20260810 11:49:20.052508 10028 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:20.052664 10028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:20.053002 10028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:20.056082 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "342d904995574391917f62aa551ade8d"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "4f3ba61ef8e17e262f35406448c15822"
server_key_iv: "9f0258a4747b1f713d911f68c5caab54"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.056923 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "342d904995574391917f62aa551ade8d"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "4f3ba61ef8e17e262f35406448c15822"
server_key_iv: "9f0258a4747b1f713d911f68c5caab54"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.062263 10028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 11:49:20.065784 10043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:20.067193 10028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:49:20.067397 10028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "342d904995574391917f62aa551ade8d"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "4f3ba61ef8e17e262f35406448c15822"
server_key_iv: "9f0258a4747b1f713d911f68c5caab54"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.067626 10028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:20.090768 10028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:20.091732 10028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:20.092053 10028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:20.112046 10028 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:36285
I20260810 11:49:20.112085 10097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:36285 every 8 connection(s)
I20260810 11:49:20.113843 10028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:20.118709 10098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:20.122784 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10028
I20260810 11:49:20.123006 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:20.123507 20854 external_mini_cluster.cc:1428] Setting key 65118c34d2cb540c051f6a4e62eb7208
I20260810 11:49:20.133827 10098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: Bootstrap starting.
I20260810 11:49:20.138111 10098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:20.139478 10098 log.cc:824] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:20.140097 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:20.125693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.20.93.190:36285)
0810 11:49:20.126884 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:20.126910 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:20.126969 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:49:20.128835 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:20.128873 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:20.129633 (+ 760us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:20.130181 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:20.130799 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.132854 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.134502 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.135620 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.135851 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:20.137054 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:20.137067 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:20.137087 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:20.137886 (+ 799us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:20.138035 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:20.138067 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:20.138209 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:20.138576 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 11:49:20.138935 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":375,"threads_started":1}
I20260810 11:49:20.142874 10098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: No bootstrap required, opened a new log
I20260810 11:49:20.152323 10098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } }
I20260810 11:49:20.152902 10098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:20.153039 10098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342d904995574391917f62aa551ade8d, State: Initialized, Role: FOLLOWER
I20260810 11:49:20.153749 10098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } }
I20260810 11:49:20.154009 10098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:20.154178 10098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:20.154372 10098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:20.156818 10098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } }
I20260810 11:49:20.157519 10098 leader_election.cc:302] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 342d904995574391917f62aa551ade8d; no voters:
I20260810 11:49:20.158293 10098 leader_election.cc:288] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:20.158384 10104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:20.159204 10104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [term 1 LEADER]: Becoming Leader. State: Replica: 342d904995574391917f62aa551ade8d, State: Running, Role: LEADER
I20260810 11:49:20.159773 10104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } }
I20260810 11:49:20.160605 10098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:20.162930 10106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 342d904995574391917f62aa551ade8d. Latest consensus state: current_term: 1 leader_uuid: "342d904995574391917f62aa551ade8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } } }
I20260810 11:49:20.163493 10106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:20.164237 10105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "342d904995574391917f62aa551ade8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342d904995574391917f62aa551ade8d" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 36285 } } }
I20260810 11:49:20.164707 10105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:20.167167 10113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:20.173185 10113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:20.182714 10113 catalog_manager.cc:1347] Generated new cluster ID: d48149c9ebed473c967907210dc3a220
I20260810 11:49:20.182843 10113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:20.203549 10113 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:20.204885 10113 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:20.213994 10113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: Generated new TSK 0
I20260810 11:49:20.214952 10113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:20.225013 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.129:0
--local_ip_for_outbound_sockets=127.20.93.129
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--builtin_ntp_servers=127.20.93.148:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:49:20.586936 10124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:20.587232 10124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:20.587320 10124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:20.594848 10124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:20.594967 10124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:20.595162 10124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.129
I20260810 11:49:20.604629 10124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.93.129
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.129
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:20.606091 10124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:20.607960 10124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:20.621392 10131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:20.621493 10129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:20.621610 10133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:20.622602 10124 server_base.cc:1034] running on GCE node
I20260810 11:49:20.623291 10124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:20.624364 10124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:20.625694 10124 hybrid_clock.cc:648] HybridClock initialized: now 1786362560625632 us; error 51 us; skew 500 ppm
I20260810 11:49:20.626042 10124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:20.628703 10124 webserver.cc:460] Webserver started at http://127.20.93.129:38079/ using document root <none> and password file <none>
I20260810 11:49:20.629463 10124 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:20.629637 10124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:20.629987 10124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:20.633057 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data/instance:
uuid: "845bc88f5d7c4fd0b1f88ab6f422666c"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "d40e97e28432d1bdc9453d0065a20d2e"
server_key_iv: "9b53f3ca31fa4b372afd4d4f78b355ab"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.634091 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance:
uuid: "845bc88f5d7c4fd0b1f88ab6f422666c"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "d40e97e28432d1bdc9453d0065a20d2e"
server_key_iv: "9b53f3ca31fa4b372afd4d4f78b355ab"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.639752 10124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:20.643592 10139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:20.645030 10124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:49:20.645219 10124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal
uuid: "845bc88f5d7c4fd0b1f88ab6f422666c"
format_stamp: "Formatted at 2026-08-10 11:49:20 on dist-test-slave-njxd"
server_key: "d40e97e28432d1bdc9453d0065a20d2e"
server_key_iv: "9b53f3ca31fa4b372afd4d4f78b355ab"
server_key_version: "encryptionkey@0"
I20260810 11:49:20.645457 10124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:20.664033 10124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:20.664973 10124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:20.665305 10124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:20.666440 10124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:20.668032 10124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:20.668165 10124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:20.668309 10124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:20.668398 10124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:20.703100 10124 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.129:46445
I20260810 11:49:20.703145 10262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.129:46445 every 8 connection(s)
I20260810 11:49:20.704493 10124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/data/info.pb
I20260810 11:49:20.710970 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10124
I20260810 11:49:20.711160 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal/instance
I20260810 11:49:20.711730 20854 external_mini_cluster.cc:1428] Setting key fe24bdc8ae18fb97e36f172a4f882704
I20260810 11:49:20.714833 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.130:0
--local_ip_for_outbound_sockets=127.20.93.130
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--builtin_ntp_servers=127.20.93.148:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:20.719323 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:20.707251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.129:57495 (local address 127.20.93.190:36285)
0810 11:49:20.707762 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:20.707767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:20.709196 (+ 1429us) server_negotiation.cc:407] Connection header received
0810 11:49:20.711599 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:20.711604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:20.711705 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:20.711839 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:20.713196 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.714229 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.715944 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.716741 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.716882 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:20.718321 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:20.718330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:20.718338 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:20.718366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:20.718418 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:20.718436 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:20.718533 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:20.718750 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 11:49:20.718867 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":270,"threads_started":1}
I20260810 11:49:20.720177 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:20.707883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36285 (local address 127.20.93.129:57495)
0810 11:49:20.708993 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:20.709039 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:20.711219 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:20.712131 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:20.712140 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:20.712239 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:20.712970 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:20.712996 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.714399 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.714403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:20.715786 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:20.715799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:20.716972 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:20.716977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:20.717205 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:20.717967 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:20.717970 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:20.718013 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:20.718113 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:20.718535 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:20.718546 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:20.718554 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:49:20.718710 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 11:49:20.719032 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":373,"threads_started":1}
I20260810 11:49:20.721869 10263 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36285
I20260810 11:49:20.722221 10263 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:20.723268 10263 heartbeater.cc:511] Master 127.20.93.190:36285 requested a full tablet report, sending...
I20260810 11:49:20.725653 10061 ts_manager.cc:194] Registered new tserver with Master: 845bc88f5d7c4fd0b1f88ab6f422666c (127.20.93.129:46445)
I20260810 11:49:20.727142 10061 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.129:57495
W20260810 11:49:21.060770 10267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:21.061076 10267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:21.061223 10267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:21.068730 10267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:21.068840 10267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:21.069018 10267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.130
I20260810 11:49:21.082419 10267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.93.130
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.130
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:21.083783 10267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:21.085716 10267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:21.099215 10272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:21.099269 10273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:21.099305 10275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:21.100117 10267 server_base.cc:1034] running on GCE node
I20260810 11:49:21.100699 10267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:21.101763 10267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:21.102967 10267 hybrid_clock.cc:648] HybridClock initialized: now 1786362561102905 us; error 61 us; skew 500 ppm
I20260810 11:49:21.103291 10267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:21.105636 10267 webserver.cc:460] Webserver started at http://127.20.93.130:46771/ using document root <none> and password file <none>
I20260810 11:49:21.106343 10267 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:21.106460 10267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:21.106796 10267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:21.109935 10267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data/instance:
uuid: "a654d6cc3df14b529828d823ba495c7a"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "0f41b4c502b34befd8647d4791eaab47"
server_key_iv: "52fba47f12213742539fd3422ce6cc9c"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.110760 10267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance:
uuid: "a654d6cc3df14b529828d823ba495c7a"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "0f41b4c502b34befd8647d4791eaab47"
server_key_iv: "52fba47f12213742539fd3422ce6cc9c"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.117399 10267 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:49:21.121928 10281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.123498 10267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:49:21.123661 10267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal
uuid: "a654d6cc3df14b529828d823ba495c7a"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "0f41b4c502b34befd8647d4791eaab47"
server_key_iv: "52fba47f12213742539fd3422ce6cc9c"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.123896 10267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:21.148296 10267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:21.149113 10267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:21.149472 10267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:21.150673 10267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:21.152081 10267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:21.152166 10267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.152243 10267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:21.152297 10267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.186313 10267 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.130:44951
I20260810 11:49:21.186405 10402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.130:44951 every 8 connection(s)
I20260810 11:49:21.187805 10267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/data/info.pb
I20260810 11:49:21.189404 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10267
I20260810 11:49:21.189618 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal/instance
I20260810 11:49:21.190182 20854 external_mini_cluster.cc:1428] Setting key 256b9eef289961c5f24e576dbbc0816d
I20260810 11:49:21.193728 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.93.131:0
--local_ip_for_outbound_sockets=127.20.93.131
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--builtin_ntp_servers=127.20.93.148:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:49:21.205111 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.191381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.130:38297 (local address 127.20.93.190:36285)
0810 11:49:21.191575 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.191581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.193311 (+ 1730us) server_negotiation.cc:407] Connection header received
0810 11:49:21.196289 (+ 2978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.196294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.196371 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.196455 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:21.197944 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.199003 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.201612 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.202262 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.202387 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.204144 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:21.204152 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:21.204159 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:21.204185 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:21.204225 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:21.204242 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:21.204324 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.204599 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.204712 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:49:21.206563 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.191949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36285 (local address 127.20.93.130:38297)
0810 11:49:21.193112 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.193167 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.195941 (+ 2774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.196728 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.196743 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.196869 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:21.197756 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.197784 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.199187 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.199195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.201426 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.201444 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.202519 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.202530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.202890 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.203741 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:21.203747 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:21.203806 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:21.203927 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:21.204346 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:21.204359 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:21.204370 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.204570 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.205036 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":350,"threads_started":1}
I20260810 11:49:21.208484 10403 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36285
I20260810 11:49:21.208849 10403 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:21.209771 10403 heartbeater.cc:511] Master 127.20.93.190:36285 requested a full tablet report, sending...
I20260810 11:49:21.211436 10061 ts_manager.cc:194] Registered new tserver with Master: a654d6cc3df14b529828d823ba495c7a (127.20.93.130:44951)
I20260810 11:49:21.212698 10061 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.130:38297
W20260810 11:49:21.549798 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:21.550096 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:21.550190 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:21.557734 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:21.557837 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:21.557945 10406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.93.131
I20260810 11:49:21.567508 10406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.93.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.93.131
--webserver_port=0
--tserver_master_addrs=127.20.93.190:36285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.93.131
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:21.568835 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:21.570809 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:21.584816 10413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:21.584769 10412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:21.584793 10415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:21.585631 10406 server_base.cc:1034] running on GCE node
I20260810 11:49:21.586305 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:21.587359 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:21.588599 10406 hybrid_clock.cc:648] HybridClock initialized: now 1786362561588535 us; error 61 us; skew 500 ppm
I20260810 11:49:21.588935 10406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:49:21.599609 10406 webserver.cc:460] Webserver started at http://127.20.93.131:36051/ using document root <none> and password file <none>
I20260810 11:49:21.600437 10406 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:21.600584 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:21.600929 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:21.603988 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data/instance:
uuid: "187fe330e09f4bf0ae34e1bcf5b3351e"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "2e80b1d912cc1ec7c0d351721e02a8e9"
server_key_iv: "eba7a1ca60a1f5b97917498b5f2db9bf"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.604797 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance:
uuid: "187fe330e09f4bf0ae34e1bcf5b3351e"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "2e80b1d912cc1ec7c0d351721e02a8e9"
server_key_iv: "eba7a1ca60a1f5b97917498b5f2db9bf"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.610440 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:49:21.614423 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.616328 10406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 11:49:21.616654 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal
uuid: "187fe330e09f4bf0ae34e1bcf5b3351e"
format_stamp: "Formatted at 2026-08-10 11:49:21 on dist-test-slave-njxd"
server_key: "2e80b1d912cc1ec7c0d351721e02a8e9"
server_key_iv: "eba7a1ca60a1f5b97917498b5f2db9bf"
server_key_version: "encryptionkey@0"
I20260810 11:49:21.616922 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:21.638902 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:21.639842 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:21.640117 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:21.641101 10406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:49:21.642781 10406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:49:21.642872 10406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.642949 10406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:49:21.643031 10406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:21.686990 10406 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.131:46483
I20260810 11:49:21.687074 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.131:46483 every 8 connection(s)
I20260810 11:49:21.689003 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/data/info.pb
I20260810 11:49:21.691061 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10406
I20260810 11:49:21.691262 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal/instance
I20260810 11:49:21.691988 20854 external_mini_cluster.cc:1428] Setting key 04aa9bf338e634edeaf97b58342882c3
I20260810 11:49:21.708570 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.696739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:34141 (local address 127.20.93.190:36285)
0810 11:49:21.696945 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.696951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.697408 (+ 457us) server_negotiation.cc:407] Connection header received
0810 11:49:21.699614 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.699619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.699733 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.699843 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:21.701334 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.702225 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.704720 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.705417 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.705637 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.707401 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:21.707408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:21.707413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:21.707438 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:21.707477 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:21.707494 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:21.707555 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.707921 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.708069 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:49:21.709899 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.695942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:36285 (local address 127.20.93.131:34141)
0810 11:49:21.697246 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.697287 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.699219 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.700162 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.700188 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.700334 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:49:21.701091 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.701121 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.702423 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.702429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.704550 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.704565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.705738 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.705747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.706073 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.706972 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:49:21.706976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:49:21.707039 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:49:21.707160 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:49:21.707637 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:49:21.707648 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:49:21.707657 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.707855 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.708329 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":486,"threads_started":1}
I20260810 11:49:21.711874 10541 heartbeater.cc:347] Connected to a master server at 127.20.93.190:36285
I20260810 11:49:21.712383 10541 heartbeater.cc:464] Registering TS with master...
I20260810 11:49:21.713646 10541 heartbeater.cc:511] Master 127.20.93.190:36285 requested a full tablet report, sending...
I20260810 11:49:21.715471 10061 ts_manager.cc:194] Registered new tserver with Master: 187fe330e09f4bf0ae34e1bcf5b3351e (127.20.93.131:46483)
I20260810 11:49:21.716728 10061 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.93.131:34141
I20260810 11:49:21.725538 20854 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:49:21.730332 10263 heartbeater.cc:503] Master 127.20.93.190:36285 was elected leader, sending a full tablet report...
I20260810 11:49:21.737020 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.730162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.20.93.190:36285)
0810 11:49:21.730443 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.730451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.731184 (+ 733us) server_negotiation.cc:407] Connection header received
0810 11:49:21.731399 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.731405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.731502 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.731612 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:21.732211 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.732928 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.734429 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.734872 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.735023 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.736215 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:21.736221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:21.736226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:21.736254 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:21.736296 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:21.736312 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:21.736412 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.736569 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.736673 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:49:21.740962 10061 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:49:21.748390 10061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:49:21.770751 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.761903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.131:46483 (local address 127.0.0.1:58706)
0810 11:49:21.762702 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.762742 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.762935 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.764149 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.764156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.764188 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:21.764548 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.764567 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.766596 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.766603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.768140 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.768154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.769970 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.769977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.770221 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.770270 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.770361 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.770503 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":355,"threads_started":1}
I20260810 11:49:21.772157 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.762096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.20.93.131:46483)
0810 11:49:21.763471 (+ 1375us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.763481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.763522 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:49:21.763630 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.763639 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.763918 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.764136 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:21.764745 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.766443 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.768346 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.769699 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.771370 (+ 1671us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.771610 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.771669 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.771808 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":1025,"threads_started":1}
I20260810 11:49:21.781042 10475 tablet_service.cc:1467] Processing CreateTablet for tablet 171469e38ddc4d9985386be864bc3a24 (DEFAULT_TABLE table=test-table [id=549e20d842334cf59383d1e6e8896987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:21.782377 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.763226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44951 (local address 127.0.0.1:39088)
0810 11:49:21.763798 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.763820 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.763928 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.775586 (+ 11658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.775590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.775618 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:21.775919 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.775928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.777689 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.777694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.779759 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.779773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.781323 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.781329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.781685 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.781733 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.781843 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.781999 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":48,"thread_start_us":247,"threads_started":1}
I20260810 11:49:21.782652 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.763767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.20.93.130:44951)
0810 11:49:21.774726 (+ 10959us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.774737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.774778 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:49:21.774916 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.774928 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.775286 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.775561 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:21.776135 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.777515 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.780021 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.781100 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.781280 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.781553 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.781974 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.782175 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10718,"thread_start_us":12010,"threads_started":1}
I20260810 11:49:21.783047 10475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171469e38ddc4d9985386be864bc3a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:21.785260 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.763816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56322)
0810 11:49:21.771040 (+ 7224us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.771068 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.771178 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.777363 (+ 6185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.777368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.777398 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:21.777675 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.777687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.779614 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.779620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.781528 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.781544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.784381 (+ 2837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.784388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.784594 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.784629 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.784722 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.784887 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7110,"mutex_wait_us":27,"thread_start_us":12358,"threads_started":1}
I20260810 11:49:21.785363 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.764468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.20.93.129:46445)
0810 11:49:21.776812 (+ 12344us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.776821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.776852 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:49:21.776954 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.776960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.777155 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.777374 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:21.777861 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.779412 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.781736 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.784157 (+ 2421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.784320 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.784605 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.784813 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.784987 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12140,"thread_start_us":12208,"threads_started":1}
I20260810 11:49:21.786495 10336 tablet_service.cc:1467] Processing CreateTablet for tablet 171469e38ddc4d9985386be864bc3a24 (DEFAULT_TABLE table=test-table [id=549e20d842334cf59383d1e6e8896987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:21.788695 10336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171469e38ddc4d9985386be864bc3a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:21.789398 10194 tablet_service.cc:1467] Processing CreateTablet for tablet 171469e38ddc4d9985386be864bc3a24 (DEFAULT_TABLE table=test-table [id=549e20d842334cf59383d1e6e8896987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:49:21.790889 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171469e38ddc4d9985386be864bc3a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:21.796452 10561 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Bootstrap starting.
I20260810 11:49:21.799876 10561 tablet_bootstrap.cc:654] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:21.800933 10561 log.cc:824] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:21.804225 10561 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: No bootstrap required, opened a new log
I20260810 11:49:21.804523 10561 ts_tablet_manager.cc:1397] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 11:49:21.805828 10563 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Bootstrap starting.
I20260810 11:49:21.809937 10563 tablet_bootstrap.cc:654] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:21.810477 10564 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Bootstrap starting.
I20260810 11:49:21.811463 10563 log.cc:824] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:21.814014 10564 tablet_bootstrap.cc:654] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:21.815305 10564 log.cc:824] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:21.815467 10563 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: No bootstrap required, opened a new log
I20260810 11:49:21.815330 10561 raft_consensus.cc:348] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.815773 10563 ts_tablet_manager.cc:1397] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 11:49:21.815817 10561 raft_consensus.cc:374] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:21.815919 10561 raft_consensus.cc:729] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 187fe330e09f4bf0ae34e1bcf5b3351e, State: Initialized, Role: FOLLOWER
I20260810 11:49:21.816610 10561 consensus_queue.cc:260] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.821310 10541 heartbeater.cc:503] Master 127.20.93.190:36285 was elected leader, sending a full tablet report...
I20260810 11:49:21.822068 10561 ts_tablet_manager.cc:1428] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260810 11:49:21.823580 10564 tablet_bootstrap.cc:492] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: No bootstrap required, opened a new log
I20260810 11:49:21.823890 10564 ts_tablet_manager.cc:1397] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260810 11:49:21.827280 10563 raft_consensus.cc:348] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.827780 10563 raft_consensus.cc:374] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:21.827893 10563 raft_consensus.cc:729] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a654d6cc3df14b529828d823ba495c7a, State: Initialized, Role: FOLLOWER
I20260810 11:49:21.828732 10563 consensus_queue.cc:260] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.830096 10403 heartbeater.cc:503] Master 127.20.93.190:36285 was elected leader, sending a full tablet report...
I20260810 11:49:21.830766 10563 ts_tablet_manager.cc:1428] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 11:49:21.833985 10564 raft_consensus.cc:348] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.834467 10564 raft_consensus.cc:374] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:21.834613 10564 raft_consensus.cc:729] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845bc88f5d7c4fd0b1f88ab6f422666c, State: Initialized, Role: FOLLOWER
I20260810 11:49:21.835321 10564 consensus_queue.cc:260] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.837251 10564 ts_tablet_manager.cc:1428] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260810 11:49:21.940135 10404 tablet.cc:2369] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:49:21.945214 10542 tablet.cc:2369] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:21.951499 10567 raft_consensus.cc:482] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:49:21.951779 10567 raft_consensus.cc:504] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.953548 10567 leader_election.cc:288] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a654d6cc3df14b529828d823ba495c7a (127.20.93.130:44951), 845bc88f5d7c4fd0b1f88ab6f422666c (127.20.93.129:46445)
W20260810 11:49:21.960137 10264 tablet.cc:2369] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:49:21.960549 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.953987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:38087 (local address 127.20.93.130:44951)
0810 11:49:21.954319 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.954328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.954460 (+ 132us) server_negotiation.cc:407] Connection header received
0810 11:49:21.954751 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.954759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.954882 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.955005 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:21.955559 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.956527 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.958788 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.959554 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.959735 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.959843 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.960095 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.960232 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260810 11:49:21.960601 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.953965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.130:44951 (local address 127.20.93.131:38087)
0810 11:49:21.954313 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.954336 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.954547 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.955083 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.955089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.955126 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:21.955376 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.955392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.956681 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.956687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.958622 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.958635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.959720 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.959726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.959907 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.959932 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.960060 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.960204 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":21}
I20260810 11:49:21.961707 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.954727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.93.131:46197 (local address 127.20.93.129:46445)
0810 11:49:21.954946 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:21.954954 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:21.955489 (+ 535us) server_negotiation.cc:407] Connection header received
0810 11:49:21.955664 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:21.955671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:21.955762 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:21.955865 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:49:21.956376 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.957288 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.959380 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.960515 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.960670 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.960781 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:21.961288 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 11:49:21.961452 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:49:21.961730 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:21.954389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.20.93.131:46197)
0810 11:49:21.955351 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:21.955368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:49:21.955522 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:49:21.955883 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:49:21.955886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:49:21.955907 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:49:21.956207 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.956214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.957456 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.957485 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.959185 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:49:21.959198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:49:21.960899 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:49:21.960905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:49:21.961111 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:21.961136 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:49:21.961219 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 11:49:21.961342 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":21,"thread_start_us":570,"threads_started":1}
I20260810 11:49:21.961903 10357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171469e38ddc4d9985386be864bc3a24" candidate_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a654d6cc3df14b529828d823ba495c7a" is_pre_election: true
I20260810 11:49:21.962524 10357 raft_consensus.cc:2454] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 187fe330e09f4bf0ae34e1bcf5b3351e in term 0.
I20260810 11:49:21.962853 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171469e38ddc4d9985386be864bc3a24" candidate_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" is_pre_election: true
I20260810 11:49:21.963297 10424 leader_election.cc:302] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 187fe330e09f4bf0ae34e1bcf5b3351e, a654d6cc3df14b529828d823ba495c7a; no voters:
I20260810 11:49:21.963394 10215 raft_consensus.cc:2454] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 187fe330e09f4bf0ae34e1bcf5b3351e in term 0.
I20260810 11:49:21.963799 10567 raft_consensus.cc:2781] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:49:21.963980 10567 raft_consensus.cc:482] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:49:21.964113 10567 raft_consensus.cc:3037] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:21.966542 10567 raft_consensus.cc:504] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.967382 10567 leader_election.cc:288] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [CANDIDATE]: Term 1 election: Requested vote from peers a654d6cc3df14b529828d823ba495c7a (127.20.93.130:44951), 845bc88f5d7c4fd0b1f88ab6f422666c (127.20.93.129:46445)
I20260810 11:49:21.967985 10357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171469e38ddc4d9985386be864bc3a24" candidate_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a654d6cc3df14b529828d823ba495c7a"
I20260810 11:49:21.968067 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171469e38ddc4d9985386be864bc3a24" candidate_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c"
I20260810 11:49:21.968267 10357 raft_consensus.cc:3037] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:21.968297 10215 raft_consensus.cc:3037] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:21.970700 10215 raft_consensus.cc:2454] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 187fe330e09f4bf0ae34e1bcf5b3351e in term 1.
I20260810 11:49:21.970733 10357 raft_consensus.cc:2454] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 187fe330e09f4bf0ae34e1bcf5b3351e in term 1.
I20260810 11:49:21.971227 10424 leader_election.cc:302] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 187fe330e09f4bf0ae34e1bcf5b3351e, 845bc88f5d7c4fd0b1f88ab6f422666c; no voters:
I20260810 11:49:21.971560 10567 raft_consensus.cc:2781] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:21.972098 10567 raft_consensus.cc:686] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 1 LEADER]: Becoming Leader. State: Replica: 187fe330e09f4bf0ae34e1bcf5b3351e, State: Running, Role: LEADER
I20260810 11:49:21.972697 10567 consensus_queue.cc:237] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } }
I20260810 11:49:21.977749 10061 catalog_manager.cc:5495] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e reported cstate change: term changed from 0 to 1, leader changed from <none> to 187fe330e09f4bf0ae34e1bcf5b3351e (127.20.93.131). New cstate: current_term: 1 leader_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "187fe330e09f4bf0ae34e1bcf5b3351e" member_type: VOTER last_known_addr { host: "127.20.93.131" port: 46483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 } health_report { overall_health: UNKNOWN } } }
I20260810 11:49:22.017187 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.011646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.20.93.131:46483)
0810 11:49:22.011896 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:22.011904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:22.011955 (+ 51us) server_negotiation.cc:407] Connection header received
0810 11:49:22.012162 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:22.012166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:22.012258 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:22.012396 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:49:22.012753 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.013501 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.015106 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.015517 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.015618 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:22.016038 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:49:22.016560 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:49:22.016700 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:22.016813 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 11:49:22.016876 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:49:22.025719 10357 raft_consensus.cc:1261] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 1 FOLLOWER]: Refusing update from remote peer 187fe330e09f4bf0ae34e1bcf5b3351e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:22.025713 10215 raft_consensus.cc:1261] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 1 FOLLOWER]: Refusing update from remote peer 187fe330e09f4bf0ae34e1bcf5b3351e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:49:22.026744 10571 consensus_queue.cc:1035] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a654d6cc3df14b529828d823ba495c7a" member_type: VOTER last_known_addr { host: "127.20.93.130" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:22.027163 10567 consensus_queue.cc:1035] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [LEADER]: Connected to new peer: Peer: permanent_uuid: "845bc88f5d7c4fd0b1f88ab6f422666c" member_type: VOTER last_known_addr { host: "127.20.93.129" port: 46445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:49:22.039436 10574 mvcc.cc:205] Tried to move back new op lower bound from 7316941054046990336 to 7316941053844766720. Current Snapshot: MvccSnapshot[applied={T|T < 7316941054046990336}]
I20260810 11:49:22.042138 10575 mvcc.cc:205] Tried to move back new op lower bound from 7316941054046990336 to 7316941053844766720. Current Snapshot: MvccSnapshot[applied={T|T < 7316941054046990336}]
I20260810 11:49:22.054766 10060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:22.055049 10060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:49:22.058892 10060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 342d904995574391917f62aa551ade8d: Sending DeleteTablet for 3 replicas of tablet 171469e38ddc4d9985386be864bc3a24
I20260810 11:49:22.060071 10475 tablet_service.cc:1514] Processing DeleteTablet for tablet 171469e38ddc4d9985386be864bc3a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:22 UTC) from {username='slave'} at 127.0.0.1:58706
I20260810 11:49:22.060298 10336 tablet_service.cc:1514] Processing DeleteTablet for tablet 171469e38ddc4d9985386be864bc3a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:22 UTC) from {username='slave'} at 127.0.0.1:39088
I20260810 11:49:22.060695 10194 tablet_service.cc:1514] Processing DeleteTablet for tablet 171469e38ddc4d9985386be864bc3a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:49:22 UTC) from {username='slave'} at 127.0.0.1:56322
I20260810 11:49:22.061930 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10124
I20260810 11:49:22.071036 10585 tablet_replica.cc:330] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: stopping tablet replica
I20260810 11:49:22.071245 10583 tablet_replica.cc:330] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: stopping tablet replica
I20260810 11:49:22.071570 10585 raft_consensus.cc:2229] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:22.071650 10583 raft_consensus.cc:2229] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:49:22.072175 10585 raft_consensus.cc:2258] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:22.072345 10583 raft_consensus.cc:2258] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:22.074848 10583 ts_tablet_manager.cc:1905] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:22.082908 10583 ts_tablet_manager.cc:1918] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:22.083125 10583 log.cc:1196] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-2/wal/wals/171469e38ddc4d9985386be864bc3a24
I20260810 11:49:22.083734 10583 ts_tablet_manager.cc:1939] T 171469e38ddc4d9985386be864bc3a24 P 187fe330e09f4bf0ae34e1bcf5b3351e: Deleting consensus metadata
I20260810 11:49:22.085930 10046 catalog_manager.cc:4841] TS 187fe330e09f4bf0ae34e1bcf5b3351e (127.20.93.131:46483): tablet 171469e38ddc4d9985386be864bc3a24 (table test-table [id=549e20d842334cf59383d1e6e8896987]) successfully deleted
I20260810 11:49:22.087847 10585 ts_tablet_manager.cc:1905] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:22.091465 10584 tablet_replica.cc:330] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: stopping tablet replica
I20260810 11:49:22.091985 10584 raft_consensus.cc:2229] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:49:22.092614 10584 raft_consensus.cc:2258] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:49:22.094287 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.064254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.20.93.129:46445)
0810 11:49:22.064506 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:22.064534 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:22.075540 (+ 11006us) server_negotiation.cc:407] Connection header received
0810 11:49:22.075883 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:22.075892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:22.076059 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:22.076183 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:22.077133 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.078768 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.088515 (+ 9747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.089554 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.089745 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:22.092159 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:22.092172 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:22.092191 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:22.093051 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:22.093129 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:22.093159 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:22.093272 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:22.093710 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 11:49:22.093858 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:49:22.095507 10584 ts_tablet_manager.cc:1905] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:49:22.095868 10585 ts_tablet_manager.cc:1918] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:22.095912 10257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:22.096742 10585 log.cc:1196] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-0/wal/wals/171469e38ddc4d9985386be864bc3a24
I20260810 11:49:22.105530 10584 ts_tablet_manager.cc:1918] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:49:22.105868 10584 log.cc:1196] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Deleting WAL directory at /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786362228785499-20854-0/minicluster-data/ts-1/wal/wals/171469e38ddc4d9985386be864bc3a24
I20260810 11:49:22.106849 10584 ts_tablet_manager.cc:1939] T 171469e38ddc4d9985386be864bc3a24 P a654d6cc3df14b529828d823ba495c7a: Deleting consensus metadata
I20260810 11:49:22.109203 10046 catalog_manager.cc:4841] TS a654d6cc3df14b529828d823ba495c7a (127.20.93.130:44951): tablet 171469e38ddc4d9985386be864bc3a24 (table test-table [id=549e20d842334cf59383d1e6e8896987]) successfully deleted
I20260810 11:49:22.204305 10585 ts_tablet_manager.cc:1939] T 171469e38ddc4d9985386be864bc3a24 P 845bc88f5d7c4fd0b1f88ab6f422666c: Deleting consensus metadata
I20260810 11:49:22.204439 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10124
W20260810 11:49:22.226536 10046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.93.129:46445 (error 108)
I20260810 11:49:22.228263 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10267
I20260810 11:49:22.228645 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.228176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56340)
0810 11:49:22.228388 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.228488 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:49:22.229158 10046 catalog_manager.cc:4584] TS 845bc88f5d7c4fd0b1f88ab6f422666c (127.20.93.129:46445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 171469e38ddc4d9985386be864bc3a24: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
I20260810 11:49:22.235770 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.229181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.20.93.130:44951)
0810 11:49:22.229393 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:22.229399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:22.229419 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:49:22.229633 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:22.229646 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:22.229746 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:22.229861 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:22.230315 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.231221 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.232534 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.232966 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.233111 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:22.234315 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:22.234324 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:22.234339 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:22.234954 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:22.235008 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:22.235028 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:22.235164 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:22.235307 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 11:49:22.235418 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:49:22.236557 10398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:22.283293 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.282622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56346)
0810 11:49:22.282924 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.283035 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":70}
I20260810 11:49:22.284696 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.284258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56360)
0810 11:49:22.284434 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.284551 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 11:49:22.328775 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10267
I20260810 11:49:22.345558 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.344917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56370)
0810 11:49:22.345218 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.345347 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 11:49:22.346946 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.346469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56372)
0810 11:49:22.346661 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.346758 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 11:49:22.349892 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10406
I20260810 11:49:22.357934 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.350757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58718 (local address 127.20.93.131:46483)
0810 11:49:22.350949 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:22.350957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:22.350981 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:49:22.351055 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:22.351060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:22.351176 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:22.351304 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:22.351678 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.352856 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.354147 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:22.354841 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:22.354990 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:22.356051 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:22.356065 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:22.356085 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:22.356969 (+ 884us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:22.357051 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:22.357090 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:22.357218 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:22.357335 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 11:49:22.357443 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 11:49:22.358716 10536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:22.428297 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.427807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56376)
0810 11:49:22.428041 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.428131 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 11:49:22.429420 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.428998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56382)
0810 11:49:22.429205 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.429282 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 11:49:22.482059 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10406
I20260810 11:49:22.503327 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10028
I20260810 11:49:22.504446 10093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:22.614673 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:22.614037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.129:46445 (local address 127.0.0.1:56392)
0810 11:49:22.614270 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 11:49:22.614413 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.129:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 11:49:22.614920 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10028
2026-08-10T11:49:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3062 ms)
[----------] 6 tests from EncryptionPolicyTest (23325 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:22 dist-test-slave-njxd krb5kdc[10594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:22 dist-test-slave-njxd krb5kdc[10594](info): set up 2 sockets
Aug 10 11:49:22 dist-test-slave-njxd krb5kdc[10594](info): commencing operation
krb5kdc: starting...
W20260810 11:49:23.805466 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:24.148910 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:24.509296 10610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:24.509702 10610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:24.509822 10610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:24.517308 10610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:24.517431 10610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:24.517479 10610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:24.517549 10610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:24.517659 10610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:24.528009 10610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:24.529359 10610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:24.530835 10610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:24.543103 10618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:24.543186 10615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:24.543154 10616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:24.543759 10610 server_base.cc:1034] running on GCE node
I20260810 11:49:24.544679 10610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:24.546296 10610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:24.547499 10610 hybrid_clock.cc:648] HybridClock initialized: now 1786362564547450 us; error 39 us; skew 500 ppm
I20260810 11:49:24.547889 10610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:24.551167 10610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:24.552706 10610 webserver.cc:460] Webserver started at http://127.0.0.1:44947/ using document root <none> and password file <none>
I20260810 11:49:24.553416 10610 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:24.553566 10610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:24.553956 10610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:24.556963 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "7ae93939af1a4cd1bbc032ba1f3597bf"
format_stamp: "Formatted at 2026-08-10 11:49:24 on dist-test-slave-njxd"
server_key: "3e3992820e7e6ad13776a6f73cde3218"
server_key_iv: "08be4755ebe183d68da87109f7b31b92"
server_key_version: "encryptionkey@0"
I20260810 11:49:24.557823 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "7ae93939af1a4cd1bbc032ba1f3597bf"
format_stamp: "Formatted at 2026-08-10 11:49:24 on dist-test-slave-njxd"
server_key: "3e3992820e7e6ad13776a6f73cde3218"
server_key_iv: "08be4755ebe183d68da87109f7b31b92"
server_key_version: "encryptionkey@0"
I20260810 11:49:24.563095 10610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:49:24.566776 10625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:24.568094 10610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:49:24.568290 10610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "7ae93939af1a4cd1bbc032ba1f3597bf"
format_stamp: "Formatted at 2026-08-10 11:49:24 on dist-test-slave-njxd"
server_key: "3e3992820e7e6ad13776a6f73cde3218"
server_key_iv: "08be4755ebe183d68da87109f7b31b92"
server_key_version: "encryptionkey@0"
I20260810 11:49:24.568552 10610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:24.591290 10610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:24.595553 10610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:24.595857 10610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:24.615566 10610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43947
I20260810 11:49:24.615584 10682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43947 every 8 connection(s)
I20260810 11:49:24.617179 10610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:24.620813 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10610
I20260810 11:49:24.621065 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:24.621362 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:24.621882 20854 external_mini_cluster.cc:1428] Setting key 1413b8a8245440fb1d5c8cdd16f41832
I20260810 11:49:24.634210 10683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf: Bootstrap starting.
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:49:24.637897 10683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:24.639091 10683 log.cc:824] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:24.642247 10683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf: No bootstrap required, opened a new log
I20260810 11:49:24.646142 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:24.624018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42043 (local address 127.0.0.1:43947)
0810 11:49:24.624884 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:24.624901 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:24.624947 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:49:24.626674 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:24.626697 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:24.627122 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:24.627566 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:24.629128 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:24.630500 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:24.631809 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:24.632527 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:24.632728 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:24.637567 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:24.637630 (+ 63us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:24.637657 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:24.637701 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:24.641097 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:24.642137 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:24.642144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:24.642151 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:24.642384 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:24.642914 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:24.642919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:24.642922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:24.643505 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:24.643854 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:24.644410 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 11:49:24.644888 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":402,"threads_started":1}
I20260810 11:49:24.650771 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:24.650998 10683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260810 11:49:24.651401 10683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:24.651528 10683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae93939af1a4cd1bbc032ba1f3597bf, State: Initialized, Role: FOLLOWER
I20260810 11:49:24.652139 10683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260810 11:49:24.652386 10683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:24.652522 10683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:24.652710 10683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:24.655507 10683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260810 11:49:24.656154 10683 leader_election.cc:302] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ae93939af1a4cd1bbc032ba1f3597bf; no voters:
I20260810 11:49:24.656924 10683 leader_election.cc:288] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:24.657053 10693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:24.659117 10683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:24.665012 10693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [term 1 LEADER]: Becoming Leader. State: Replica: 7ae93939af1a4cd1bbc032ba1f3597bf, State: Running, Role: LEADER
I20260810 11:49:24.665724 10693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260810 11:49:24.669348 10694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260810 11:49:24.669255 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ae93939af1a4cd1bbc032ba1f3597bf. Latest consensus state: current_term: 1 leader_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae93939af1a4cd1bbc032ba1f3597bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260810 11:49:24.669675 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:24.669675 10694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:24.674616 10703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:24.678989 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:24.656198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33967 (local address 127.0.0.1:43947)
0810 11:49:24.656410 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:24.656416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:24.657068 (+ 652us) server_negotiation.cc:407] Connection header received
0810 11:49:24.657325 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:24.657330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:24.657439 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:24.657566 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:24.660433 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:24.662132 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:24.664776 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:24.665827 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:24.666012 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:24.670015 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:24.670026 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:24.670034 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:24.670090 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:24.675072 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:24.676118 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:24.676123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:24.676126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:24.676283 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:24.677030 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:24.677034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:24.677037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:24.677532 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:24.677869 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:24.678326 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 11:49:24.678545 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:49:24.680459 10703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:24.689402 10703 catalog_manager.cc:1347] Generated new cluster ID: d1f5f06de43b42298f594a9e2574d0ef
I20260810 11:49:24.689529 10703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:24.704775 10703 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:24.706630 10703 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:24.721314 10703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ae93939af1a4cd1bbc032ba1f3597bf: Generated new TSK 0
I20260810 11:49:24.722496 10703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:24.732560 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10610
I20260810 11:49:24.734242 10678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:24.828668 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10610
W20260810 11:49:24.849964 10592 connection.cc:535] client connection to 127.0.0.1:43947 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43947): Connection reset by peer (error 104)
2026-08-10T11:49:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10716](info): set up 2 sockets
Aug 10 11:49:24 dist-test-slave-njxd krb5kdc[10716](info): commencing operation
krb5kdc: starting...
W20260810 11:49:26.022709 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:26 dist-test-slave-njxd krb5kdc[10716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:26.355022 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:26.709698 10729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:26.710001 10729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:26.710093 10729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:26.717304 10729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:26.717417 10729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:26.717476 10729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:26.717545 10729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:26.717651 10729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:26.727742 10729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:26.729074 10729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:26.730520 10729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:26.742690 10734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:26.742729 10735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:26.742776 10738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:26.743860 10729 server_base.cc:1034] running on GCE node
I20260810 11:49:26.744660 10729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:26.746203 10729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:26.747418 10729 hybrid_clock.cc:648] HybridClock initialized: now 1786362566747372 us; error 41 us; skew 500 ppm
I20260810 11:49:26.747757 10729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:26 dist-test-slave-njxd krb5kdc[10716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:26.755056 10729 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:26.756711 10729 webserver.cc:460] Webserver started at http://127.0.0.1:33663/ using document root <none> and password file <none>
I20260810 11:49:26.757436 10729 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:26.757625 10729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:26.757956 10729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:26.761023 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "2a6466317fd24bc3b48e068c0f6818d0"
format_stamp: "Formatted at 2026-08-10 11:49:26 on dist-test-slave-njxd"
server_key: "8c0c95ed1e69316c6ecd29409decca39"
server_key_iv: "839c9b3d61ac020d03976da2a4a3ddbc"
server_key_version: "encryptionkey@0"
I20260810 11:49:26.761916 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "2a6466317fd24bc3b48e068c0f6818d0"
format_stamp: "Formatted at 2026-08-10 11:49:26 on dist-test-slave-njxd"
server_key: "8c0c95ed1e69316c6ecd29409decca39"
server_key_iv: "839c9b3d61ac020d03976da2a4a3ddbc"
server_key_version: "encryptionkey@0"
I20260810 11:49:26.767321 10729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:49:26.771090 10746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:26.772517 10729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:49:26.772718 10729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "2a6466317fd24bc3b48e068c0f6818d0"
format_stamp: "Formatted at 2026-08-10 11:49:26 on dist-test-slave-njxd"
server_key: "8c0c95ed1e69316c6ecd29409decca39"
server_key_iv: "839c9b3d61ac020d03976da2a4a3ddbc"
server_key_version: "encryptionkey@0"
I20260810 11:49:26.772933 10729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:26.788489 10729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:26.792645 10729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:26.792951 10729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:26.812297 10729 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39267
I20260810 11:49:26.812335 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39267 every 8 connection(s)
I20260810 11:49:26.813752 10729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:26.817400 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10729
I20260810 11:49:26.817611 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:26.817721 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:26.818145 20854 external_mini_cluster.cc:1428] Setting key a626bfc734431b4644e7036ab7c6e013
I20260810 11:49:26.833633 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:26.820002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37217 (local address 127.0.0.1:39267)
0810 11:49:26.820914 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:26.820936 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:26.820994 (+ 58us) server_negotiation.cc:407] Connection header received
0810 11:49:26.823031 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:26.823061 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:26.823625 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:26.824105 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:26.824663 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:26.826744 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:26.828124 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:26.829186 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:26.829372 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:26.830652 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:26.830663 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:26.830683 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:26.831477 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:26.831585 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:26.831622 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:26.831799 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:26.832075 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 11:49:26.832255 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":377,"threads_started":1}
I20260810 11:49:26.834075 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0: Bootstrap starting.
I20260810 11:49:26.837603 10807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:26.837625 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:26.838676 10807 log.cc:824] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:26.842015 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0: No bootstrap required, opened a new log
I20260810 11:49:26.849741 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:26.842641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48143 (local address 127.0.0.1:39267)
0810 11:49:26.842823 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:26.842833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:26.843209 (+ 376us) server_negotiation.cc:407] Connection header received
0810 11:49:26.843383 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:26.843388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:26.843467 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:26.843586 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:26.844118 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:26.845342 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:26.846782 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:26.847539 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:26.847699 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:26.848837 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:26.848843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:26.848847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:26.848872 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:26.848936 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:26.848958 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:26.849089 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:26.849252 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 11:49:26.849315 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:49:26.852229 10807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260810 11:49:26.852658 10807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:26.852782 10807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6466317fd24bc3b48e068c0f6818d0, State: Initialized, Role: FOLLOWER
I20260810 11:49:26.853483 10807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260810 11:49:26.853799 10807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:26.853955 10807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:26.854141 10807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:26.856488 10807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260810 11:49:26.857023 10807 leader_election.cc:302] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a6466317fd24bc3b48e068c0f6818d0; no voters:
I20260810 11:49:26.857666 10807 leader_election.cc:288] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:26.857831 10817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:26.858574 10817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2a6466317fd24bc3b48e068c0f6818d0, State: Running, Role: LEADER
I20260810 11:49:26.859169 10817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260810 11:49:26.860077 10807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:26.862329 10819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a6466317fd24bc3b48e068c0f6818d0. Latest consensus state: current_term: 1 leader_uuid: "2a6466317fd24bc3b48e068c0f6818d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } } }
I20260810 11:49:26.862815 10819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:26.863696 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a6466317fd24bc3b48e068c0f6818d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6466317fd24bc3b48e068c0f6818d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } } }
I20260810 11:49:26.864085 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:26.866716 10827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:26.872229 10827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:26.881947 10827 catalog_manager.cc:1347] Generated new cluster ID: 2313cacd57e14ddd9b21a9bc9263777e
I20260810 11:49:26.882084 10827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:26.901466 10827 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:26.902797 10827 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:26.911816 10827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a6466317fd24bc3b48e068c0f6818d0: Generated new TSK 0
I20260810 11:49:26.912623 10827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:26.924530 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10729
W20260810 11:49:26.925724 10801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37217
W20260810 11:49:26.926638 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10729: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:26.926811 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10729
W20260810 11:49:26.949066 10714 connection.cc:535] client connection to 127.0.0.1:39267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39267): Connection reset by peer (error 104)
2026-08-10T11:49:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:27 dist-test-slave-njxd krb5kdc[10842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:27 dist-test-slave-njxd krb5kdc[10842](info): set up 2 sockets
Aug 10 11:49:27 dist-test-slave-njxd krb5kdc[10842](info): commencing operation
krb5kdc: starting...
W20260810 11:49:28.126121 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:28 dist-test-slave-njxd krb5kdc[10842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:28.462787 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:28.830209 10855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:28.830544 10855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:28.830641 10855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:28.838209 10855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:28.838346 10855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:28.838395 10855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:28.838457 10855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:28.838495 10855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:28.849057 10855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:28.850472 10855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:28.851926 10855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:28.864599 10860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:28.864645 10861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:28.864707 10863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:28.865254 10855 server_base.cc:1034] running on GCE node
I20260810 11:49:28.866245 10855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:28.867715 10855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:28.868958 10855 hybrid_clock.cc:648] HybridClock initialized: now 1786362568868913 us; error 40 us; skew 500 ppm
I20260810 11:49:28.869333 10855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:28 dist-test-slave-njxd krb5kdc[10842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362568, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:28.872605 10855 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:28.874275 10855 webserver.cc:460] Webserver started at http://127.0.0.1:38127/ using document root <none> and password file <none>
I20260810 11:49:28.874998 10855 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:28.875155 10855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:28.875485 10855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:28.878640 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "87264efec0f64a8d98b71215b78594f8"
format_stamp: "Formatted at 2026-08-10 11:49:28 on dist-test-slave-njxd"
server_key: "2e36f565a15656b6138c15412c338ecb"
server_key_iv: "4cbba210ad295de0f21c539f69fa4370"
server_key_version: "encryptionkey@0"
I20260810 11:49:28.879456 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "87264efec0f64a8d98b71215b78594f8"
format_stamp: "Formatted at 2026-08-10 11:49:28 on dist-test-slave-njxd"
server_key: "2e36f565a15656b6138c15412c338ecb"
server_key_iv: "4cbba210ad295de0f21c539f69fa4370"
server_key_version: "encryptionkey@0"
I20260810 11:49:28.885113 10855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:28.888832 10870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:28.890326 10855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:28.890522 10855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "87264efec0f64a8d98b71215b78594f8"
format_stamp: "Formatted at 2026-08-10 11:49:28 on dist-test-slave-njxd"
server_key: "2e36f565a15656b6138c15412c338ecb"
server_key_iv: "4cbba210ad295de0f21c539f69fa4370"
server_key_version: "encryptionkey@0"
I20260810 11:49:28.890753 10855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:28.896759 10855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:28.901197 10855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:28.901508 10855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:28.920959 10855 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44511
I20260810 11:49:28.921000 10923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44511 every 8 connection(s)
I20260810 11:49:28.922554 10855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:28.926544 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10855
I20260810 11:49:28.926759 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:28.926801 10924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:28.927325 20854 external_mini_cluster.cc:1428] Setting key 041cdf4f8b7c7c9c39a63f6b0619a4e1
I20260810 11:49:28.935917 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:28.929341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59147 (local address 127.0.0.1:44511)
0810 11:49:28.930798 (+ 1457us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:28.930810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:28.930860 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:49:28.932417 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:28.932439 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:28.932805 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:28.933195 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:28.934312 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:28.934319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:28.934333 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:28.934347 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:28.934417 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:28.934437 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:28.934623 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:28.934737 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 11:49:28.934944 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1197,"thread_start_us":893,"threads_started":1}
I20260810 11:49:28.940058 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:28.941756 10924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8: Bootstrap starting.
I20260810 11:49:28.946054 10924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:28.947505 10924 log.cc:824] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:28.948724 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:28.945531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39473 (local address 127.0.0.1:44511)
0810 11:49:28.945853 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:28.945863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:28.946280 (+ 417us) server_negotiation.cc:407] Connection header received
0810 11:49:28.946467 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:28.946473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:28.946555 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:28.946686 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:28.947850 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:28.947856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:28.947860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:28.947879 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:28.947928 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:28.947949 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:28.948088 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:28.948341 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 11:49:28.948405 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:49:28.951103 10924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8: No bootstrap required, opened a new log
I20260810 11:49:28.958737 10924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260810 11:49:28.959177 10924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:28.959296 10924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87264efec0f64a8d98b71215b78594f8, State: Initialized, Role: FOLLOWER
I20260810 11:49:28.959897 10924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260810 11:49:28.960134 10924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:28.960294 10924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:28.960459 10924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:28.962721 10924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260810 11:49:28.963225 10924 leader_election.cc:302] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87264efec0f64a8d98b71215b78594f8; no voters:
I20260810 11:49:28.963867 10924 leader_election.cc:288] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:28.964049 10936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:28.964727 10936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [term 1 LEADER]: Becoming Leader. State: Replica: 87264efec0f64a8d98b71215b78594f8, State: Running, Role: LEADER
I20260810 11:49:28.965314 10936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260810 11:49:28.966001 10924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:28.968485 10938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87264efec0f64a8d98b71215b78594f8. Latest consensus state: current_term: 1 leader_uuid: "87264efec0f64a8d98b71215b78594f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } } }
I20260810 11:49:28.968886 10938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:28.969343 10937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87264efec0f64a8d98b71215b78594f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87264efec0f64a8d98b71215b78594f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } } }
I20260810 11:49:28.969700 10937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:28.970768 10943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:28.975998 10943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:28.985781 10943 catalog_manager.cc:1347] Generated new cluster ID: 13355039ca6f4abeaea98521a6198fa0
I20260810 11:49:28.985953 10943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:28.997797 10943 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:28.998389 10943 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:29.008747 10943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87264efec0f64a8d98b71215b78594f8: Generated new TSK 0
I20260810 11:49:29.009677 10943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:29.023380 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10855
W20260810 11:49:29.024695 10919 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59147
W20260810 11:49:29.025441 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10855: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:29.025627 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10855
2026-08-10T11:49:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:29 dist-test-slave-njxd krb5kdc[10958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:29 dist-test-slave-njxd krb5kdc[10958](info): set up 2 sockets
Aug 10 11:49:29 dist-test-slave-njxd krb5kdc[10958](info): commencing operation
krb5kdc: starting...
W20260810 11:49:30.232895 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:30 dist-test-slave-njxd krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:30.585309 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:30.948311 10975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:30.948729 10975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:30.948915 10975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:30.956686 10975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:30.956862 10975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:30.956975 10975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:30.957085 10975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:30.957168 10975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:30.967888 10975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:30.969367 10975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:30.970880 10975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:30.983922 10980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:30.983896 10981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:30.984104 10983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:30.984768 10975 server_base.cc:1034] running on GCE node
I20260810 11:49:30.985706 10975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:30.987228 10975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:30.988451 10975 hybrid_clock.cc:648] HybridClock initialized: now 1786362570988408 us; error 42 us; skew 500 ppm
I20260810 11:49:30.988826 10975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:30 dist-test-slave-njxd krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362570, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:30.992164 10975 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:30.993846 10975 webserver.cc:460] Webserver started at http://127.0.0.1:40829/ using document root <none> and password file <none>
I20260810 11:49:30.994596 10975 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:30.994721 10975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:30.994985 10975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:30.998425 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "1abb7a3407434072a1c219b9cea4956c"
format_stamp: "Formatted at 2026-08-10 11:49:30 on dist-test-slave-njxd"
server_key: "3dff808e5545d396fdf91f845912fda6"
server_key_iv: "408b6aa0b6a44e1d766888075c7c8941"
server_key_version: "encryptionkey@0"
I20260810 11:49:30.999317 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "1abb7a3407434072a1c219b9cea4956c"
format_stamp: "Formatted at 2026-08-10 11:49:30 on dist-test-slave-njxd"
server_key: "3dff808e5545d396fdf91f845912fda6"
server_key_iv: "408b6aa0b6a44e1d766888075c7c8941"
server_key_version: "encryptionkey@0"
I20260810 11:49:31.005059 10975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:49:31.008791 10990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:31.010414 10975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:31.010592 10975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "1abb7a3407434072a1c219b9cea4956c"
format_stamp: "Formatted at 2026-08-10 11:49:30 on dist-test-slave-njxd"
server_key: "3dff808e5545d396fdf91f845912fda6"
server_key_iv: "408b6aa0b6a44e1d766888075c7c8941"
server_key_version: "encryptionkey@0"
I20260810 11:49:31.010787 10975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:31.026904 10975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:31.031467 10975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:31.031728 10975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:31.051215 10975 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43175
I20260810 11:49:31.051249 11045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43175 every 8 connection(s)
I20260810 11:49:31.052692 10975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:31.056768 11046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:31.059484 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 10975
I20260810 11:49:31.059682 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:31.060133 20854 external_mini_cluster.cc:1428] Setting key 17d5aaa47f6ff9bcd7d335ae7338d78c
I20260810 11:49:31.069103 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c: Bootstrap starting.
I20260810 11:49:31.074059 11046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:31.075630 11046 log.cc:824] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:49:31 dist-test-slave-njxd krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:49:31.079650 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c: No bootstrap required, opened a new log
I20260810 11:49:31.087914 11049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:31.062046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50369 (local address 127.0.0.1:43175)
0810 11:49:31.062907 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:31.062920 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:31.062963 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:49:31.064669 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:31.064692 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:31.065122 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:31.065478 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:31.067447 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:31.068774 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:31.070651 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:31.071943 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:31.072149 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:31.078280 (+ 6131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:31.078293 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:31.078314 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:31.078368 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:31.082762 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:31.083627 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:31.083637 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:31.083646 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:31.083877 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:31.084293 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:31.084299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:31.084304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:31.085078 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:31.085433 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:31.085856 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 11:49:31.086368 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":409,"threads_started":1}
I20260810 11:49:31.090883 11046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } }
I20260810 11:49:31.091297 11046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:31.091419 11046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abb7a3407434072a1c219b9cea4956c, State: Initialized, Role: FOLLOWER
I20260810 11:49:31.092092 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:31.092093 11046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } }
I20260810 11:49:31.092376 11046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:31.092521 11046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:31.092701 11046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:31.095140 11046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } }
I20260810 11:49:31.095710 11046 leader_election.cc:302] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1abb7a3407434072a1c219b9cea4956c; no voters:
I20260810 11:49:31.096321 11046 leader_election.cc:288] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:31.096521 11055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:31.098766 11046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:31.103528 11055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [term 1 LEADER]: Becoming Leader. State: Replica: 1abb7a3407434072a1c219b9cea4956c, State: Running, Role: LEADER
I20260810 11:49:31.104310 11055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } }
I20260810 11:49:31.107748 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1abb7a3407434072a1c219b9cea4956c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } } }
I20260810 11:49:31.108178 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:31.115418 11057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1abb7a3407434072a1c219b9cea4956c. Latest consensus state: current_term: 1 leader_uuid: "1abb7a3407434072a1c219b9cea4956c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1abb7a3407434072a1c219b9cea4956c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43175 } } }
I20260810 11:49:31.115872 11057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:31.117727 11068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:31.121559 11049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:31.097041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47301 (local address 127.0.0.1:43175)
0810 11:49:31.097239 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:31.097245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:31.098480 (+ 1235us) server_negotiation.cc:407] Connection header received
0810 11:49:31.098909 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:31.098914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:31.099014 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:31.099122 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:31.101650 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:31.103158 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:31.105011 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:31.105733 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:31.105878 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:31.109879 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:31.109888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:31.109896 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:31.109950 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:31.116889 (+ 6939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:31.118110 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:31.118115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:31.118119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:31.118279 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:31.118795 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:31.118802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:31.118808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:31.119283 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:31.119520 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:31.120897 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0810 11:49:31.121146 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:49:31.126441 11068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:31.136078 11068 catalog_manager.cc:1347] Generated new cluster ID: c72b435ffc774489881fff30305d6177
I20260810 11:49:31.136222 11068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:31.151389 11068 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:31.152705 11068 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:31.164003 11068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1abb7a3407434072a1c219b9cea4956c: Generated new TSK 0
I20260810 11:49:31.164867 11068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:31.171504 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 10975
I20260810 11:49:31.172704 11041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:31.267431 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 10975
2026-08-10T11:49:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:31 dist-test-slave-njxd krb5kdc[11081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:31 dist-test-slave-njxd krb5kdc[11081](info): set up 2 sockets
Aug 10 11:49:31 dist-test-slave-njxd krb5kdc[11081](info): commencing operation
krb5kdc: starting...
W20260810 11:49:32.469533 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:32 dist-test-slave-njxd krb5kdc[11081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:32.817751 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:33.188400 11095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:33.188798 11095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:33.188963 11095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:33.196563 11095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:33.196762 11095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:33.196874 11095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:33.196982 11095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:33.197069 11095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:33.207338 11095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:33.208761 11095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:33.210215 11095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:33.222621 11101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:33.222586 11105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:33.222677 11102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:33.223544 11095 server_base.cc:1034] running on GCE node
I20260810 11:49:33.224450 11095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:33.226023 11095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:33.227262 11095 hybrid_clock.cc:648] HybridClock initialized: now 1786362573227215 us; error 41 us; skew 500 ppm
I20260810 11:49:33.227630 11095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:33 dist-test-slave-njxd krb5kdc[11081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:33.236219 11095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:33.237910 11095 webserver.cc:460] Webserver started at http://127.0.0.1:45713/ using document root <none> and password file <none>
I20260810 11:49:33.238612 11095 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:33.238723 11095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:33.238981 11095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:33.242187 11095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "4fe7932bf3c64941b0cb8a57d4638bee"
format_stamp: "Formatted at 2026-08-10 11:49:33 on dist-test-slave-njxd"
server_key: "55cdc0e77b6e8a2061e162999dac7f3f"
server_key_iv: "1d4ff4144e7364332234dee66bdd61ad"
server_key_version: "encryptionkey@0"
I20260810 11:49:33.242933 11095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "4fe7932bf3c64941b0cb8a57d4638bee"
format_stamp: "Formatted at 2026-08-10 11:49:33 on dist-test-slave-njxd"
server_key: "55cdc0e77b6e8a2061e162999dac7f3f"
server_key_iv: "1d4ff4144e7364332234dee66bdd61ad"
server_key_version: "encryptionkey@0"
I20260810 11:49:33.248247 11095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:33.251839 11113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:33.253249 11095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:49:33.253386 11095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "4fe7932bf3c64941b0cb8a57d4638bee"
format_stamp: "Formatted at 2026-08-10 11:49:33 on dist-test-slave-njxd"
server_key: "55cdc0e77b6e8a2061e162999dac7f3f"
server_key_iv: "1d4ff4144e7364332234dee66bdd61ad"
server_key_version: "encryptionkey@0"
I20260810 11:49:33.253554 11095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:33.266294 11095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:33.270596 11095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:33.270855 11095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:33.290266 11095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36151
I20260810 11:49:33.290294 11167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36151 every 8 connection(s)
I20260810 11:49:33.291886 11095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:33.292389 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11095
I20260810 11:49:33.292577 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:33.293243 20854 external_mini_cluster.cc:1428] Setting key 7fe7eacd5144a00a4bcb48b3b7865515
I20260810 11:49:33.296736 11168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:33.308159 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:33.295364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33135 (local address 127.0.0.1:36151)
0810 11:49:33.296215 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:33.296227 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:33.296272 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:49:33.298119 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:33.298144 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:33.298561 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:33.298915 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:33.299631 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:33.301068 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:33.303128 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:33.303883 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:33.304048 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:33.305342 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:33.305349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:33.305364 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:33.306028 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:33.306127 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:33.306157 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:33.306256 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:33.306560 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 11:49:33.306927 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":386,"threads_started":1}
I20260810 11:49:33.310202 11168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee: Bootstrap starting.
I20260810 11:49:33.312551 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:33.314235 11168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:33.315604 11168 log.cc:824] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:33.319252 11168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee: No bootstrap required, opened a new log
I20260810 11:49:33.325368 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:33.318289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33795 (local address 127.0.0.1:36151)
0810 11:49:33.318479 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:33.318485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:33.318975 (+ 490us) server_negotiation.cc:407] Connection header received
0810 11:49:33.319191 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:33.319200 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:33.319301 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:33.319432 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:33.320056 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:33.321134 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:33.322588 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:33.323318 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:33.323474 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:33.324547 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:33.324554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:33.324559 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:33.324583 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:33.324636 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:33.324663 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:33.324754 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:33.324886 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 11:49:33.324995 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:49:33.328455 11168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260810 11:49:33.328883 11168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:33.329001 11168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fe7932bf3c64941b0cb8a57d4638bee, State: Initialized, Role: FOLLOWER
I20260810 11:49:33.329641 11168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260810 11:49:33.329881 11168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:33.330036 11168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:33.330199 11168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:33.332470 11168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260810 11:49:33.333062 11168 leader_election.cc:302] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fe7932bf3c64941b0cb8a57d4638bee; no voters:
I20260810 11:49:33.333704 11168 leader_election.cc:288] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:33.333841 11179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:33.334637 11179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [term 1 LEADER]: Becoming Leader. State: Replica: 4fe7932bf3c64941b0cb8a57d4638bee, State: Running, Role: LEADER
I20260810 11:49:33.335222 11179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260810 11:49:33.336223 11168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:33.338622 11181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fe7932bf3c64941b0cb8a57d4638bee. Latest consensus state: current_term: 1 leader_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } } }
I20260810 11:49:33.338619 11180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe7932bf3c64941b0cb8a57d4638bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } } }
I20260810 11:49:33.339114 11181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:33.339162 11180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:33.341650 11186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:33.346267 11186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:33.355674 11186 catalog_manager.cc:1347] Generated new cluster ID: 46b617b78af34cf68bb4b9bacdd1f6c3
I20260810 11:49:33.355806 11186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:33.367177 11186 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:33.368649 11186 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:33.389492 11186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fe7932bf3c64941b0cb8a57d4638bee: Generated new TSK 0
I20260810 11:49:33.390506 11186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:33.410425 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11095
W20260810 11:49:33.411717 11163 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33135
W20260810 11:49:33.412529 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11095: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:33.412740 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11095
2026-08-10T11:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2145 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:33 dist-test-slave-njxd krb5kdc[11200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:33 dist-test-slave-njxd krb5kdc[11200](info): set up 2 sockets
Aug 10 11:49:33 dist-test-slave-njxd krb5kdc[11200](info): commencing operation
krb5kdc: starting...
W20260810 11:49:34.606650 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:34 dist-test-slave-njxd krb5kdc[11200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:34.954299 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:35.327777 11213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:35.328078 11213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:35.328224 11213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:35.335845 11213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:35.336006 11213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:35.336112 11213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:35.336220 11213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:35.336293 11213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:35.346843 11213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:35.348284 11213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:35.349787 11213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:35.362419 11218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:35.362432 11219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:35.362459 11221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:35.363226 11213 server_base.cc:1034] running on GCE node
I20260810 11:49:35.364071 11213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:35.365705 11213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:35.367005 11213 hybrid_clock.cc:648] HybridClock initialized: now 1786362575366935 us; error 52 us; skew 500 ppm
I20260810 11:49:35.367476 11213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:35 dist-test-slave-njxd krb5kdc[11200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:35.371435 11213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:35.373345 11213 webserver.cc:460] Webserver started at http://127.0.0.1:44017/ using document root <none> and password file <none>
I20260810 11:49:35.374219 11213 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:35.374382 11213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:35.374738 11213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:35.378360 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a89e46761ff3452abbdfcb5f7026dff2"
format_stamp: "Formatted at 2026-08-10 11:49:35 on dist-test-slave-njxd"
server_key: "59456b8666920d6256640b9575e9def0"
server_key_iv: "c85a044785a08864fd4be3731009d6f1"
server_key_version: "encryptionkey@0"
I20260810 11:49:35.379572 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a89e46761ff3452abbdfcb5f7026dff2"
format_stamp: "Formatted at 2026-08-10 11:49:35 on dist-test-slave-njxd"
server_key: "59456b8666920d6256640b9575e9def0"
server_key_iv: "c85a044785a08864fd4be3731009d6f1"
server_key_version: "encryptionkey@0"
I20260810 11:49:35.386129 11213 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260810 11:49:35.390086 11228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:35.391649 11213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:49:35.391856 11213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a89e46761ff3452abbdfcb5f7026dff2"
format_stamp: "Formatted at 2026-08-10 11:49:35 on dist-test-slave-njxd"
server_key: "59456b8666920d6256640b9575e9def0"
server_key_iv: "c85a044785a08864fd4be3731009d6f1"
server_key_version: "encryptionkey@0"
I20260810 11:49:35.392087 11213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:35.398586 11213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:35.403293 11213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:35.403644 11213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:35.424171 11213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43497
I20260810 11:49:35.424232 11287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43497 every 8 connection(s)
I20260810 11:49:35.425843 11213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:35.430163 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:35.430565 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11213
I20260810 11:49:35.430765 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:35.431332 20854 external_mini_cluster.cc:1428] Setting key 736f41ac4cb827487c4e21bf5fc3f4da
I20260810 11:49:35.441039 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:35.433336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43801 (local address 127.0.0.1:43497)
0810 11:49:35.434435 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:35.434449 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:35.434496 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:49:35.436950 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:35.436982 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:35.437449 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:35.437952 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:35.439235 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:35.439241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:35.439256 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:35.439272 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:35.439338 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:35.439364 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:35.439553 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:35.439739 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 11:49:35.439968 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":558,"threads_started":1}
I20260810 11:49:35.445500 11288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2: Bootstrap starting.
I20260810 11:49:35.446142 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:35.450395 11288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:35.452086 11288 log.cc:824] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:35.455168 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:35.452334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54299 (local address 127.0.0.1:43497)
0810 11:49:35.452542 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:35.452549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:35.452996 (+ 447us) server_negotiation.cc:407] Connection header received
0810 11:49:35.453222 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:35.453226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:35.453281 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:35.453405 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:35.454465 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:35.454471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:35.454475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:35.454499 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:35.454550 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:35.454566 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:35.454727 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:35.454822 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 11:49:35.454892 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:49:35.455823 11288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2: No bootstrap required, opened a new log
I20260810 11:49:35.464412 11288 raft_consensus.cc:348] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260810 11:49:35.464834 11288 raft_consensus.cc:374] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:35.464913 11288 raft_consensus.cc:729] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89e46761ff3452abbdfcb5f7026dff2, State: Initialized, Role: FOLLOWER
I20260810 11:49:35.465473 11288 consensus_queue.cc:260] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260810 11:49:35.465760 11288 raft_consensus.cc:388] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:35.465875 11288 raft_consensus.cc:482] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:35.465998 11288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:35.468768 11288 raft_consensus.cc:504] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260810 11:49:35.469273 11288 leader_election.cc:302] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a89e46761ff3452abbdfcb5f7026dff2; no voters:
I20260810 11:49:35.469935 11288 leader_election.cc:288] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:35.470103 11298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:35.470811 11298 raft_consensus.cc:686] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [term 1 LEADER]: Becoming Leader. State: Replica: a89e46761ff3452abbdfcb5f7026dff2, State: Running, Role: LEADER
I20260810 11:49:35.471352 11298 consensus_queue.cc:237] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } }
I20260810 11:49:35.472146 11288 sys_catalog.cc:564] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:35.474176 11300 sys_catalog.cc:455] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a89e46761ff3452abbdfcb5f7026dff2. Latest consensus state: current_term: 1 leader_uuid: "a89e46761ff3452abbdfcb5f7026dff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } } }
I20260810 11:49:35.474627 11300 sys_catalog.cc:458] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:35.476025 11299 sys_catalog.cc:455] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a89e46761ff3452abbdfcb5f7026dff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a89e46761ff3452abbdfcb5f7026dff2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43497 } } }
I20260810 11:49:35.476922 11299 sys_catalog.cc:458] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:35.476935 11304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:35.482378 11304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:35.483206 11316 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:49:35.492156 11304 catalog_manager.cc:1347] Generated new cluster ID: a6dd073b8b8540ed998b89a15ed17663
I20260810 11:49:35.492297 11304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:35.502569 11304 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:35.503108 11304 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:35.514497 11304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a89e46761ff3452abbdfcb5f7026dff2: Generated new TSK 0
I20260810 11:49:35.515409 11304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:35.523492 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11213
W20260810 11:49:35.525084 11283 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43801
W20260810 11:49:35.525945 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11213: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:35.526121 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11213
2026-08-10T11:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:35 dist-test-slave-njxd krb5kdc[11321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:35 dist-test-slave-njxd krb5kdc[11321](info): set up 2 sockets
Aug 10 11:49:35 dist-test-slave-njxd krb5kdc[11321](info): commencing operation
krb5kdc: starting...
W20260810 11:49:36.726639 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.008s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:36 dist-test-slave-njxd krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:37.082787 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:37.451131 11335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:37.451486 11335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:37.451653 11335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:37.459398 11335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:37.459542 11335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:37.459667 11335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:37.459755 11335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:37.459822 11335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:37.470736 11335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:37.472223 11335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:37.473757 11335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:37.486614 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:37.487190 11342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:37.487710 11341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:37.488054 11335 server_base.cc:1034] running on GCE node
I20260810 11:49:37.488981 11335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:37.490674 11335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:37.491923 11335 hybrid_clock.cc:648] HybridClock initialized: now 1786362577491863 us; error 51 us; skew 500 ppm
I20260810 11:49:37.492306 11335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:37 dist-test-slave-njxd krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362577, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:37.495859 11335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:37.497475 11335 webserver.cc:460] Webserver started at http://127.0.0.1:43429/ using document root <none> and password file <none>
I20260810 11:49:37.498251 11335 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:37.498366 11335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:37.498631 11335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:37.502012 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "c4552ed83ab34b57a10c258b5686eea0"
format_stamp: "Formatted at 2026-08-10 11:49:37 on dist-test-slave-njxd"
server_key: "c429143dde16bc32dd9b06a6b3070e46"
server_key_iv: "02b28408ac64b8a793cc4010ad4d1ffd"
server_key_version: "encryptionkey@0"
I20260810 11:49:37.502797 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "c4552ed83ab34b57a10c258b5686eea0"
format_stamp: "Formatted at 2026-08-10 11:49:37 on dist-test-slave-njxd"
server_key: "c429143dde16bc32dd9b06a6b3070e46"
server_key_iv: "02b28408ac64b8a793cc4010ad4d1ffd"
server_key_version: "encryptionkey@0"
I20260810 11:49:37.508659 11335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:37.512903 11351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:37.514703 11335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:49:37.514942 11335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "c4552ed83ab34b57a10c258b5686eea0"
format_stamp: "Formatted at 2026-08-10 11:49:37 on dist-test-slave-njxd"
server_key: "c429143dde16bc32dd9b06a6b3070e46"
server_key_iv: "02b28408ac64b8a793cc4010ad4d1ffd"
server_key_version: "encryptionkey@0"
I20260810 11:49:37.515174 11335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:37.539366 11335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:37.544476 11335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:37.544857 11335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:37.565711 11335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39015
I20260810 11:49:37.565810 11405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39015 every 8 connection(s)
I20260810 11:49:37.567343 11335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:37.568972 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11335
I20260810 11:49:37.569175 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:37.569782 20854 external_mini_cluster.cc:1428] Setting key ee033e17f43c9618f7b12c8c992d246c
I20260810 11:49:37.572502 11406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:49:37 dist-test-slave-njxd krb5kdc[11321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:49:37.587828 11406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0: Bootstrap starting.
I20260810 11:49:37.592640 11406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:37.594022 11406 log.cc:824] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:37.597481 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:37.571787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:52301 (local address 127.0.0.1:39015)
0810 11:49:37.572722 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:37.572737 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:37.572784 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:49:37.574649 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:37.574682 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:37.575148 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:37.575584 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:37.577799 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:37.579294 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:37.581303 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:37.582172 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:37.582328 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:37.587825 (+ 5497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:37.587838 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:37.587862 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:37.587927 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:37.592174 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:37.593347 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:37.593356 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:37.593366 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:37.593636 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:37.594287 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:37.594295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:37.594301 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:37.595204 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:37.595563 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:37.596004 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 11:49:37.596366 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":395,"threads_started":1}
I20260810 11:49:37.597656 11406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0: No bootstrap required, opened a new log
I20260810 11:49:37.601987 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:37.608134 11406 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260810 11:49:37.608732 11406 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:37.608834 11406 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4552ed83ab34b57a10c258b5686eea0, State: Initialized, Role: FOLLOWER
I20260810 11:49:37.609510 11406 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260810 11:49:37.609858 11406 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:37.610011 11406 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:37.610221 11406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:37.612727 11406 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260810 11:49:37.613364 11406 leader_election.cc:302] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4552ed83ab34b57a10c258b5686eea0; no voters:
I20260810 11:49:37.614143 11406 leader_election.cc:288] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:37.614267 11421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:37.615074 11421 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [term 1 LEADER]: Becoming Leader. State: Replica: c4552ed83ab34b57a10c258b5686eea0, State: Running, Role: LEADER
I20260810 11:49:37.615805 11421 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260810 11:49:37.618220 11421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4552ed83ab34b57a10c258b5686eea0. Latest consensus state: current_term: 1 leader_uuid: "c4552ed83ab34b57a10c258b5686eea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } } }
I20260810 11:49:37.618594 11421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:37.619040 11423 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4552ed83ab34b57a10c258b5686eea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4552ed83ab34b57a10c258b5686eea0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } } }
I20260810 11:49:37.619449 11423 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:37.621515 11406 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:37.621814 11425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:37.627946 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:37.607904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:49407 (local address 127.0.0.1:39015)
0810 11:49:37.608147 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:37.608153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:37.608477 (+ 324us) server_negotiation.cc:407] Connection header received
0810 11:49:37.608699 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:37.608706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:37.608815 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:37.608947 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:37.610758 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:37.611633 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:37.612954 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:37.613489 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:37.613661 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:37.617755 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:37.617766 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:37.617771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:37.617818 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:37.622143 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:37.623500 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:37.623509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:37.623515 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:37.623738 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:37.624504 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:37.624513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:37.624519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:37.625071 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:37.625461 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:37.627273 (+ 1812us) server_negotiation.cc:299] Negotiation successful
0810 11:49:37.627506 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:49:37.631268 11425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:37.649796 11425 catalog_manager.cc:1347] Generated new cluster ID: 0f34eac2e73c4ae3be9add71c492262e
I20260810 11:49:37.649976 11425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:37.669785 11425 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:37.671921 11425 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:37.686316 11425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4552ed83ab34b57a10c258b5686eea0: Generated new TSK 0
I20260810 11:49:37.687505 11425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:37.699220 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11335
I20260810 11:49:37.700476 11401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:37.796810 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11335
W20260810 11:49:37.815651 11318 connection.cc:535] client connection to 127.0.0.1:39015 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39015): Connection reset by peer (error 104)
2026-08-10T11:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:37 dist-test-slave-njxd krb5kdc[11445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:37 dist-test-slave-njxd krb5kdc[11445](info): set up 2 sockets
Aug 10 11:49:37 dist-test-slave-njxd krb5kdc[11445](info): commencing operation
krb5kdc: starting...
W20260810 11:49:38.993863 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.010s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:39 dist-test-slave-njxd krb5kdc[11445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:39.352639 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:39.720546 11458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:39.720881 11458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:39.721057 11458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:39.728516 11458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:39.728645 11458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:39.728761 11458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:39.728876 11458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:39.728958 11458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:39.739562 11458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:39.741024 11458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:39.742475 11458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:39.755111 11463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:39.755149 11466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:39.755234 11464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:39.756148 11458 server_base.cc:1034] running on GCE node
I20260810 11:49:39.757114 11458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:39.758699 11458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:39.759944 11458 hybrid_clock.cc:648] HybridClock initialized: now 1786362579759902 us; error 29 us; skew 500 ppm
I20260810 11:49:39.760310 11458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:39 dist-test-slave-njxd krb5kdc[11445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:39.768728 11458 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:39.770388 11458 webserver.cc:460] Webserver started at http://127.0.0.1:46359/ using document root <none> and password file <none>
I20260810 11:49:39.771116 11458 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:39.771227 11458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:39.771574 11458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:39.775031 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "6009cf7c95134b95bbce6dc48c82e1ec"
format_stamp: "Formatted at 2026-08-10 11:49:39 on dist-test-slave-njxd"
server_key: "4fa9fc1bece130db470679c5ab30ec20"
server_key_iv: "7813b36e9cc4fb21f8c3bc29932d74f7"
server_key_version: "encryptionkey@0"
I20260810 11:49:39.775826 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "6009cf7c95134b95bbce6dc48c82e1ec"
format_stamp: "Formatted at 2026-08-10 11:49:39 on dist-test-slave-njxd"
server_key: "4fa9fc1bece130db470679c5ab30ec20"
server_key_iv: "7813b36e9cc4fb21f8c3bc29932d74f7"
server_key_version: "encryptionkey@0"
I20260810 11:49:39.781970 11458 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:49:39.785917 11474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:39.787698 11458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:49:39.787920 11458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "6009cf7c95134b95bbce6dc48c82e1ec"
format_stamp: "Formatted at 2026-08-10 11:49:39 on dist-test-slave-njxd"
server_key: "4fa9fc1bece130db470679c5ab30ec20"
server_key_iv: "7813b36e9cc4fb21f8c3bc29932d74f7"
server_key_version: "encryptionkey@0"
I20260810 11:49:39.788146 11458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:39.809365 11458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:39.814448 11458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:39.814818 11458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:39.835194 11458 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36909
I20260810 11:49:39.835239 11534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36909 every 8 connection(s)
I20260810 11:49:39.836745 11458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:39.838027 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11458
I20260810 11:49:39.838240 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:39.838701 20854 external_mini_cluster.cc:1428] Setting key 6583d631c6cb1af16d2c53ef811ac60a
I20260810 11:49:39.841666 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:39.855675 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:39.840753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:34195 (local address 127.0.0.1:36909)
0810 11:49:39.841564 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:39.841621 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:39.841688 (+ 67us) server_negotiation.cc:407] Connection header received
0810 11:49:39.844126 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:39.844158 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:39.844716 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:39.845251 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:39.845888 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:39.848050 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:39.849552 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:39.850685 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:39.850881 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:39.852313 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:39.852324 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:39.852347 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:39.853181 (+ 834us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:39.853286 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:39.853324 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:39.853512 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:39.853948 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 11:49:39.854164 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":330,"threads_started":1}
I20260810 11:49:39.857009 11535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec: Bootstrap starting.
I20260810 11:49:39.860250 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:39.860739 11535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:39.861899 11535 log.cc:824] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:39.865350 11535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec: No bootstrap required, opened a new log
I20260810 11:49:39.873420 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:39.865896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:60519 (local address 127.0.0.1:36909)
0810 11:49:39.866107 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:39.866115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:39.866590 (+ 475us) server_negotiation.cc:407] Connection header received
0810 11:49:39.866764 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:39.866771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:39.866861 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:39.866979 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:39.867514 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:39.868748 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:39.870321 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:39.871063 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:39.871214 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:39.872540 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:39.872548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:39.872558 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:39.872583 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:39.872650 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:39.872669 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:39.872794 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:39.873012 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 11:49:39.873077 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:49:39.875927 11535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260810 11:49:39.876372 11535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:39.876502 11535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6009cf7c95134b95bbce6dc48c82e1ec, State: Initialized, Role: FOLLOWER
I20260810 11:49:39.877249 11535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260810 11:49:39.877560 11535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:39.877794 11535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:39.877970 11535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:39.880314 11535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260810 11:49:39.880919 11535 leader_election.cc:302] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6009cf7c95134b95bbce6dc48c82e1ec; no voters:
I20260810 11:49:39.881557 11535 leader_election.cc:288] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:39.881815 11546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:39.882593 11546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [term 1 LEADER]: Becoming Leader. State: Replica: 6009cf7c95134b95bbce6dc48c82e1ec, State: Running, Role: LEADER
I20260810 11:49:39.883280 11546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260810 11:49:39.883963 11535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:39.886821 11548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6009cf7c95134b95bbce6dc48c82e1ec. Latest consensus state: current_term: 1 leader_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } } }
I20260810 11:49:39.886813 11547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6009cf7c95134b95bbce6dc48c82e1ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } } }
I20260810 11:49:39.887450 11548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:39.887459 11547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:39.889647 11553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:39.896549 11553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:39.906677 11553 catalog_manager.cc:1347] Generated new cluster ID: 88c5fb2bf7584622a820896ded914fcd
I20260810 11:49:39.906862 11553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:39.930173 11553 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:39.931631 11553 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:39.941102 11553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6009cf7c95134b95bbce6dc48c82e1ec: Generated new TSK 0
I20260810 11:49:39.942212 11553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:39.960299 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11458
W20260810 11:49:39.961634 11530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.106:34195
W20260810 11:49:39.962574 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11458: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:39.962759 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11458
W20260810 11:49:39.985140 11443 connection.cc:535] client connection to 127.0.0.1:36909 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36909): Connection reset by peer (error 104)
2026-08-10T11:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:40 dist-test-slave-njxd krb5kdc[11570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:40 dist-test-slave-njxd krb5kdc[11570](info): set up 2 sockets
Aug 10 11:49:40 dist-test-slave-njxd krb5kdc[11570](info): commencing operation
krb5kdc: starting...
W20260810 11:49:41.162933 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:41 dist-test-slave-njxd krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:41.521114 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:41.905907 11584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:41.906282 11584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:41.906445 11584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:41.914141 11584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:41.914331 11584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:41.914438 11584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:41.914577 11584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:41.914656 11584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:41.925120 11584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:41.926718 11584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:41.928161 11584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:41.941390 11589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:41.941335 11593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:41.941448 11590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:41.942780 11584 server_base.cc:1034] running on GCE node
I20260810 11:49:41.943707 11584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:41.945331 11584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:41.946587 11584 hybrid_clock.cc:648] HybridClock initialized: now 1786362581946530 us; error 49 us; skew 500 ppm
I20260810 11:49:41.946981 11584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:41 dist-test-slave-njxd krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:41.950456 11584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:41.952131 11584 webserver.cc:460] Webserver started at http://127.0.0.1:39097/ using document root <none> and password file <none>
I20260810 11:49:41.952854 11584 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:41.952972 11584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:41.953276 11584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:41.956712 11584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "e6c8c839cafa401a9672851ea0f03ae7"
format_stamp: "Formatted at 2026-08-10 11:49:41 on dist-test-slave-njxd"
server_key: "90482b7b31dcf74f1dc8af8dbb135467"
server_key_iv: "6f32cd6894f5c99b7ebe7cd9f75dc4fd"
server_key_version: "encryptionkey@0"
I20260810 11:49:41.957677 11584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "e6c8c839cafa401a9672851ea0f03ae7"
format_stamp: "Formatted at 2026-08-10 11:49:41 on dist-test-slave-njxd"
server_key: "90482b7b31dcf74f1dc8af8dbb135467"
server_key_iv: "6f32cd6894f5c99b7ebe7cd9f75dc4fd"
server_key_version: "encryptionkey@0"
I20260810 11:49:41.963912 11584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:49:41.968061 11601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:41.969872 11584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:41.970062 11584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "e6c8c839cafa401a9672851ea0f03ae7"
format_stamp: "Formatted at 2026-08-10 11:49:41 on dist-test-slave-njxd"
server_key: "90482b7b31dcf74f1dc8af8dbb135467"
server_key_iv: "6f32cd6894f5c99b7ebe7cd9f75dc4fd"
server_key_version: "encryptionkey@0"
I20260810 11:49:41.970268 11584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:41.976883 11584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:41.981767 11584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:41.982057 11584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:42.002935 11584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44161
I20260810 11:49:42.003036 11656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44161 every 8 connection(s)
I20260810 11:49:42.004562 11584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:42.007936 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11584
I20260810 11:49:42.008131 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:42.008616 20854 external_mini_cluster.cc:1428] Setting key ba6201511bf6dd6537e285a791397e4d
I20260810 11:49:42.008958 11657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:42.017838 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:42.010546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:59337 (local address 127.0.0.1:44161)
0810 11:49:42.011360 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:42.011372 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:42.011414 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:49:42.013936 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:42.013963 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:42.014361 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:42.014778 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:42.016026 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:42.016032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:42.016049 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:42.016065 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:42.016136 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:42.016166 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:42.016354 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:42.016464 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 11:49:42.016729 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":361,"threads_started":1}
I20260810 11:49:42.022481 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:42.023968 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7: Bootstrap starting.
I20260810 11:49:42.028028 11657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:42.029233 11657 log.cc:824] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:42.030411 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:42.027770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:41549 (local address 127.0.0.1:44161)
0810 11:49:42.027991 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:42.027999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:42.028211 (+ 212us) server_negotiation.cc:407] Connection header received
0810 11:49:42.028452 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:42.028456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:42.028518 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:42.028640 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:42.029708 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:42.029714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:42.029719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:42.029745 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:42.029787 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:42.029806 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:42.029926 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:42.030056 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 11:49:42.030130 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:49:42.032615 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7: No bootstrap required, opened a new log
I20260810 11:49:42.041452 11657 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } }
I20260810 11:49:42.042019 11657 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:42.042119 11657 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c8c839cafa401a9672851ea0f03ae7, State: Initialized, Role: FOLLOWER
I20260810 11:49:42.042846 11657 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } }
I20260810 11:49:42.043170 11657 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:42.043375 11657 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:42.043622 11657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:42.046418 11657 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } }
I20260810 11:49:42.047014 11657 leader_election.cc:302] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6c8c839cafa401a9672851ea0f03ae7; no voters:
I20260810 11:49:42.047701 11657 leader_election.cc:288] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:42.047860 11669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:42.048602 11669 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [term 1 LEADER]: Becoming Leader. State: Replica: e6c8c839cafa401a9672851ea0f03ae7, State: Running, Role: LEADER
I20260810 11:49:42.049204 11669 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } }
I20260810 11:49:42.050184 11657 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:42.052505 11670 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6c8c839cafa401a9672851ea0f03ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } } }
I20260810 11:49:42.052452 11671 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6c8c839cafa401a9672851ea0f03ae7. Latest consensus state: current_term: 1 leader_uuid: "e6c8c839cafa401a9672851ea0f03ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6c8c839cafa401a9672851ea0f03ae7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44161 } } }
I20260810 11:49:42.054105 11671 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:42.054075 11670 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:42.055945 11678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:42.061689 11678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:42.072671 11678 catalog_manager.cc:1347] Generated new cluster ID: 714c84f668b34ae699aae06930ecb4f3
I20260810 11:49:42.072834 11678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:42.095016 11678 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:42.095503 11678 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:42.106738 11678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6c8c839cafa401a9672851ea0f03ae7: Generated new TSK 0
I20260810 11:49:42.107636 11678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:42.117436 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11584
W20260810 11:49:42.118718 11651 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.106:59337
W20260810 11:49:42.119481 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11584: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:42.119670 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11584
2026-08-10T11:49:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:42 dist-test-slave-njxd krb5kdc[11691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:42 dist-test-slave-njxd krb5kdc[11691](info): set up 2 sockets
Aug 10 11:49:42 dist-test-slave-njxd krb5kdc[11691](info): commencing operation
krb5kdc: starting...
W20260810 11:49:43.330086 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:43 dist-test-slave-njxd krb5kdc[11691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:43.677459 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:44.050274 11705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:44.050604 11705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:44.050712 11705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:44.058597 11705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:44.058745 11705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:44.058795 11705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:44.058851 11705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:44.058897 11705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:44.069614 11705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:44.071080 11705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:44.072670 11705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:44.086611 11713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:44.086673 11710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:44.086663 11711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:44.087656 11705 server_base.cc:1034] running on GCE node
I20260810 11:49:44.088496 11705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:44.090167 11705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:44.091382 11705 hybrid_clock.cc:648] HybridClock initialized: now 1786362584091331 us; error 45 us; skew 500 ppm
I20260810 11:49:44.091765 11705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:44 dist-test-slave-njxd krb5kdc[11691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:44.095417 11705 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:44.097100 11705 webserver.cc:460] Webserver started at http://127.0.0.1:41989/ using document root <none> and password file <none>
I20260810 11:49:44.097934 11705 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:44.098090 11705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:44.098433 11705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:44.101698 11705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "e21c7265451146fca3189a5174bd297c"
format_stamp: "Formatted at 2026-08-10 11:49:44 on dist-test-slave-njxd"
server_key: "27f9b6235dd730576b0ea1c2a165cc9d"
server_key_iv: "2f917a5858db4f8872613ffbf6a10d69"
server_key_version: "encryptionkey@0"
I20260810 11:49:44.102769 11705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "e21c7265451146fca3189a5174bd297c"
format_stamp: "Formatted at 2026-08-10 11:49:44 on dist-test-slave-njxd"
server_key: "27f9b6235dd730576b0ea1c2a165cc9d"
server_key_iv: "2f917a5858db4f8872613ffbf6a10d69"
server_key_version: "encryptionkey@0"
I20260810 11:49:44.108729 11705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:49:44.112884 11721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:44.114526 11705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:44.114759 11705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "e21c7265451146fca3189a5174bd297c"
format_stamp: "Formatted at 2026-08-10 11:49:44 on dist-test-slave-njxd"
server_key: "27f9b6235dd730576b0ea1c2a165cc9d"
server_key_iv: "2f917a5858db4f8872613ffbf6a10d69"
server_key_version: "encryptionkey@0"
I20260810 11:49:44.115000 11705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:44.153481 11705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:44.158448 11705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:44.158783 11705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:44.180055 11705 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46419
I20260810 11:49:44.180086 11777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46419 every 8 connection(s)
I20260810 11:49:44.181628 11705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:44.184567 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11705
I20260810 11:49:44.184779 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:44.185252 20854 external_mini_cluster.cc:1428] Setting key 0dd39c0977fd1a7d41248be88b4fe6b7
I20260810 11:49:44.186172 11778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:49:44 dist-test-slave-njxd krb5kdc[11691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:49:44.202152 11778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c: Bootstrap starting.
I20260810 11:49:44.206732 11778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:44.208112 11778 log.cc:824] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:44.212100 11778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c: No bootstrap required, opened a new log
I20260810 11:49:44.213052 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:44.187327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:55739 (local address 127.0.0.1:46419)
0810 11:49:44.188199 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:44.188218 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:44.188272 (+ 54us) server_negotiation.cc:407] Connection header received
0810 11:49:44.190447 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:44.190481 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:44.191152 (+ 671us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:44.191691 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:44.193273 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:44.195526 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:44.196930 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:44.198085 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:44.198295 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:44.203136 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:44.203148 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:44.203169 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:44.203222 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:44.207468 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:44.208467 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:44.208477 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:44.208486 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:44.208765 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:44.209462 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:44.209466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:44.209468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:44.210338 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:44.210666 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:44.211179 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 11:49:44.211543 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":351,"threads_started":1}
I20260810 11:49:44.217428 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:44.221663 11778 raft_consensus.cc:348] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260810 11:49:44.222128 11778 raft_consensus.cc:374] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:44.222255 11778 raft_consensus.cc:729] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21c7265451146fca3189a5174bd297c, State: Initialized, Role: FOLLOWER
I20260810 11:49:44.223119 11778 consensus_queue.cc:260] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260810 11:49:44.223435 11778 raft_consensus.cc:388] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:44.223646 11778 raft_consensus.cc:482] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:44.223886 11778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:44.227099 11778 raft_consensus.cc:504] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260810 11:49:44.227789 11778 leader_election.cc:302] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e21c7265451146fca3189a5174bd297c; no voters:
I20260810 11:49:44.228588 11778 leader_election.cc:288] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:44.228709 11789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:44.229959 11789 raft_consensus.cc:686] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [term 1 LEADER]: Becoming Leader. State: Replica: e21c7265451146fca3189a5174bd297c, State: Running, Role: LEADER
I20260810 11:49:44.230732 11789 consensus_queue.cc:237] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260810 11:49:44.231145 11778 sys_catalog.cc:564] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:44.239090 11791 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e21c7265451146fca3189a5174bd297c. Latest consensus state: current_term: 1 leader_uuid: "e21c7265451146fca3189a5174bd297c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } } }
I20260810 11:49:44.239557 11791 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:44.240211 11790 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e21c7265451146fca3189a5174bd297c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e21c7265451146fca3189a5174bd297c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } } }
I20260810 11:49:44.240543 11790 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:44.243993 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:44.222556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:54887 (local address 127.0.0.1:46419)
0810 11:49:44.222927 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:44.222934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:44.223295 (+ 361us) server_negotiation.cc:407] Connection header received
0810 11:49:44.223498 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:44.223502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:44.223606 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:44.223701 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:44.225705 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:44.226485 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:44.228436 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:44.229221 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:44.230474 (+ 1253us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:44.233387 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:44.233396 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:44.233404 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:44.233446 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:44.238427 (+ 4981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:44.239763 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:44.239769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:44.239772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:44.240013 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:44.240753 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:44.240760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:44.240766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:44.241211 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:44.242854 (+ 1643us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:44.243098 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 11:49:44.243376 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260810 11:49:44.246960 11799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:44.253998 11799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:44.264938 11799 catalog_manager.cc:1347] Generated new cluster ID: 6a4e769e8aa44c0698dca9d5df50763f
I20260810 11:49:44.265085 11799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:44.282985 11799 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:44.284999 11799 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:44.298540 11799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e21c7265451146fca3189a5174bd297c: Generated new TSK 0
I20260810 11:49:44.299712 11799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:44.308785 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11705
I20260810 11:49:44.310084 11772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:44.406350 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11705
W20260810 11:49:44.427743 11689 connection.cc:535] client connection to 127.0.0.1:46419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46419): Connection reset by peer (error 104)
2026-08-10T11:49:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:44 dist-test-slave-njxd krb5kdc[11813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:44 dist-test-slave-njxd krb5kdc[11813](info): set up 2 sockets
Aug 10 11:49:44 dist-test-slave-njxd krb5kdc[11813](info): commencing operation
krb5kdc: starting...
W20260810 11:49:45.618698 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:45 dist-test-slave-njxd krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:45.984799 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:46.363471 11829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:46.363826 11829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:46.364008 11829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:46.371649 11829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:46.371790 11829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:46.371891 11829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:46.372006 11829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:46.372089 11829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:46.382723 11829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:46.384267 11829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:46.385802 11829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:46.399025 11835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:46.398972 11834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:46.399087 11837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:46.399940 11829 server_base.cc:1034] running on GCE node
I20260810 11:49:46.400923 11829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:46.402652 11829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:46.403894 11829 hybrid_clock.cc:648] HybridClock initialized: now 1786362586403846 us; error 39 us; skew 500 ppm
I20260810 11:49:46.404332 11829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:46 dist-test-slave-njxd krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:46.414140 11829 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:46.416006 11829 webserver.cc:460] Webserver started at http://127.0.0.1:33867/ using document root <none> and password file <none>
I20260810 11:49:46.416815 11829 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:46.416988 11829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:46.417341 11829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:46.421280 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a149b9576a20443d8f54576bb065afda"
format_stamp: "Formatted at 2026-08-10 11:49:46 on dist-test-slave-njxd"
server_key: "7904fa3b8f7e77b978b4bdab3ee77d6b"
server_key_iv: "0b80186b9dfb4da40812fe84651aa7b7"
server_key_version: "encryptionkey@0"
I20260810 11:49:46.422297 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a149b9576a20443d8f54576bb065afda"
format_stamp: "Formatted at 2026-08-10 11:49:46 on dist-test-slave-njxd"
server_key: "7904fa3b8f7e77b978b4bdab3ee77d6b"
server_key_iv: "0b80186b9dfb4da40812fe84651aa7b7"
server_key_version: "encryptionkey@0"
I20260810 11:49:46.428211 11829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:46.432109 11845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:46.433787 11829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:49:46.433980 11829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a149b9576a20443d8f54576bb065afda"
format_stamp: "Formatted at 2026-08-10 11:49:46 on dist-test-slave-njxd"
server_key: "7904fa3b8f7e77b978b4bdab3ee77d6b"
server_key_iv: "0b80186b9dfb4da40812fe84651aa7b7"
server_key_version: "encryptionkey@0"
I20260810 11:49:46.434217 11829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:46.461268 11829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:46.466295 11829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:46.466622 11829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:46.487533 11829 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37297
I20260810 11:49:46.487583 11899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37297 every 8 connection(s)
I20260810 11:49:46.489167 11829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:46.492640 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11829
I20260810 11:49:46.492877 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:46.493353 20854 external_mini_cluster.cc:1428] Setting key 532ed011a5545d93529e978114cd5741
I20260810 11:49:46.493623 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:46.508492 11900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda: Bootstrap starting.
I20260810 11:49:46.509049 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:46.495535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:55001 (local address 127.0.0.1:37297)
0810 11:49:46.496424 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:46.496437 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:46.496484 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:49:46.498576 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:46.498599 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:46.499142 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:46.499510 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:46.500211 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:46.501748 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:46.503646 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:46.504370 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:46.504518 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:46.505955 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:46.505963 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:46.505980 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:46.506594 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:46.506685 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:46.506716 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:46.506845 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:46.507272 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 11:49:46.507509 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":406,"threads_started":1}
I20260810 11:49:46.512051 11900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:46.513329 11900 log.cc:824] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:46.513564 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:46.516410 11900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda: No bootstrap required, opened a new log
I20260810 11:49:46.526844 11900 raft_consensus.cc:348] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } }
I20260810 11:49:46.527276 11900 raft_consensus.cc:374] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:46.527441 11900 raft_consensus.cc:729] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a149b9576a20443d8f54576bb065afda, State: Initialized, Role: FOLLOWER
I20260810 11:49:46.528234 11900 consensus_queue.cc:260] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } }
I20260810 11:49:46.528575 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:46.518404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:50355 (local address 127.0.0.1:37297)
0810 11:49:46.518582 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:46.518589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:46.522220 (+ 3631us) server_negotiation.cc:407] Connection header received
0810 11:49:46.522471 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:46.522476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:46.522549 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:46.522672 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:46.523273 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:46.524084 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:46.525508 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:46.526076 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:46.526221 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:46.527617 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:46.527624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:46.527631 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:46.527658 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:46.527728 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:46.527748 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:46.527884 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:46.528098 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 11:49:46.528159 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 11:49:46.528616 11900 raft_consensus.cc:388] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:46.528975 11900 raft_consensus.cc:482] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:46.529238 11900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:46.532022 11900 raft_consensus.cc:504] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } }
I20260810 11:49:46.532610 11900 leader_election.cc:302] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a149b9576a20443d8f54576bb065afda; no voters:
I20260810 11:49:46.533252 11900 leader_election.cc:288] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:46.533416 11910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:46.535218 11910 raft_consensus.cc:686] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [term 1 LEADER]: Becoming Leader. State: Replica: a149b9576a20443d8f54576bb065afda, State: Running, Role: LEADER
I20260810 11:49:46.535873 11900 sys_catalog.cc:564] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:46.535876 11910 consensus_queue.cc:237] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } }
I20260810 11:49:46.538969 11911 sys_catalog.cc:455] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a149b9576a20443d8f54576bb065afda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } } }
I20260810 11:49:46.539430 11911 sys_catalog.cc:458] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:46.538993 11912 sys_catalog.cc:455] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [sys.catalog]: SysCatalogTable state changed. Reason: New leader a149b9576a20443d8f54576bb065afda. Latest consensus state: current_term: 1 leader_uuid: "a149b9576a20443d8f54576bb065afda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a149b9576a20443d8f54576bb065afda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37297 } } }
I20260810 11:49:46.540120 11912 sys_catalog.cc:458] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:46.540552 11918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:46.549469 11918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:46.562999 11918 catalog_manager.cc:1347] Generated new cluster ID: 5947444e3aae4b28be533e961cf38e68
I20260810 11:49:46.563148 11918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:46.584107 11918 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:46.586197 11918 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:46.597325 11918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a149b9576a20443d8f54576bb065afda: Generated new TSK 0
I20260810 11:49:46.598266 11918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:46.606225 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11829
W20260810 11:49:46.607568 11895 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.106:55001
W20260810 11:49:46.608536 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11829: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:46.608685 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11829
W20260810 11:49:46.632735 11811 connection.cc:535] client connection to 127.0.0.1:37297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37297): Connection reset by peer (error 104)
2026-08-10T11:49:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:46 dist-test-slave-njxd krb5kdc[11933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:46 dist-test-slave-njxd krb5kdc[11933](info): set up 2 sockets
Aug 10 11:49:46 dist-test-slave-njxd krb5kdc[11933](info): commencing operation
krb5kdc: starting...
W20260810 11:49:47.817168 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.007s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:48 dist-test-slave-njxd krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:49:48.171232 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:48.536784 11946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:48.537191 11946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:48.537365 11946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:48.544917 11946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:48.545120 11946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:48.545249 11946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:48.545368 11946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:48.545459 11946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:48.555876 11946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:48.557379 11946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:48.558833 11946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:48.571261 11951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:48.571336 11954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:48.571297 11952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:48.572659 11946 server_base.cc:1034] running on GCE node
I20260810 11:49:48.573496 11946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:48.575114 11946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:48.576371 11946 hybrid_clock.cc:648] HybridClock initialized: now 1786362588576325 us; error 64 us; skew 500 ppm
I20260810 11:49:48.576797 11946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:48 dist-test-slave-njxd krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:48.580008 11946 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:49:48.581542 11946 webserver.cc:460] Webserver started at http://127.0.0.1:45653/ using document root <none> and password file <none>
I20260810 11:49:48.582329 11946 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:48.582484 11946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:48.582804 11946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:48.586040 11946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a5f917016c3945c190d784dea7b4ce39"
format_stamp: "Formatted at 2026-08-10 11:49:48 on dist-test-slave-njxd"
server_key: "249ea225ab4ad8a0e43c5f8055fdf2e4"
server_key_iv: "ea870560698f64ed4a154b84bdc380f9"
server_key_version: "encryptionkey@0"
I20260810 11:49:48.586829 11946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a5f917016c3945c190d784dea7b4ce39"
format_stamp: "Formatted at 2026-08-10 11:49:48 on dist-test-slave-njxd"
server_key: "249ea225ab4ad8a0e43c5f8055fdf2e4"
server_key_iv: "ea870560698f64ed4a154b84bdc380f9"
server_key_version: "encryptionkey@0"
I20260810 11:49:48.592412 11946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:49:48.596050 11962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:48.597430 11946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:49:48.597697 11946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a5f917016c3945c190d784dea7b4ce39"
format_stamp: "Formatted at 2026-08-10 11:49:48 on dist-test-slave-njxd"
server_key: "249ea225ab4ad8a0e43c5f8055fdf2e4"
server_key_iv: "ea870560698f64ed4a154b84bdc380f9"
server_key_version: "encryptionkey@0"
I20260810 11:49:48.597915 11946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:48.603749 11946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:48.608045 11946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:48.608368 11946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:48.627097 11946 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39809
I20260810 11:49:48.627269 12017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39809 every 8 connection(s)
I20260810 11:49:48.628547 11946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:48.632454 12018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:48.637738 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 11946
I20260810 11:49:48.637944 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:48.638526 20854 external_mini_cluster.cc:1428] Setting key 0eb4880f8160f28ace1675aa7fd7d8ce
I20260810 11:49:48.644719 12018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39: Bootstrap starting.
I20260810 11:49:48.646987 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:48.640494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:57769 (local address 127.0.0.1:39809)
0810 11:49:48.641355 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:48.641367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:48.641412 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:49:48.643309 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:48.643333 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:48.643701 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:48.644040 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:48.645329 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:48.645335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:48.645353 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:48.645370 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:48.645444 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:48.645464 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:48.645711 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:48.645822 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 11:49:48.646046 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":386,"threads_started":1}
I20260810 11:49:48.648366 12018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:48.649590 12018 log.cc:824] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:48.650974 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:48.652684 12018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39: No bootstrap required, opened a new log
I20260810 11:49:48.658774 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:48.655712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:59545 (local address 127.0.0.1:39809)
0810 11:49:48.655918 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:48.655926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:48.656411 (+ 485us) server_negotiation.cc:407] Connection header received
0810 11:49:48.656568 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:48.656574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:48.656649 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:48.656767 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:48.657972 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:48.657977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:48.657983 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:48.658009 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:48.658059 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:48.658079 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:48.658211 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:48.658363 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 11:49:48.658442 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:49:48.662017 12018 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } }
I20260810 11:49:48.662452 12018 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:48.662566 12018 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f917016c3945c190d784dea7b4ce39, State: Initialized, Role: FOLLOWER
I20260810 11:49:48.663221 12018 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } }
I20260810 11:49:48.663471 12018 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:48.663672 12018 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:48.663880 12018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:48.666400 12018 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } }
I20260810 11:49:48.667055 12018 leader_election.cc:302] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5f917016c3945c190d784dea7b4ce39; no voters:
I20260810 11:49:48.667718 12018 leader_election.cc:288] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:48.667867 12028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:48.668591 12028 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [term 1 LEADER]: Becoming Leader. State: Replica: a5f917016c3945c190d784dea7b4ce39, State: Running, Role: LEADER
I20260810 11:49:48.669224 12028 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } }
I20260810 11:49:48.670133 12018 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:48.672511 12030 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5f917016c3945c190d784dea7b4ce39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } } }
I20260810 11:49:48.673260 12030 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:48.673125 12031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5f917016c3945c190d784dea7b4ce39. Latest consensus state: current_term: 1 leader_uuid: "a5f917016c3945c190d784dea7b4ce39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f917016c3945c190d784dea7b4ce39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39809 } } }
I20260810 11:49:48.673564 12031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:48.678748 12038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:48.684999 12038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:48.695282 12038 catalog_manager.cc:1347] Generated new cluster ID: dcd4742de15f4c58b9b492c3759ef04a
I20260810 11:49:48.695430 12038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:48.739110 12038 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:48.739655 12038 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:48.753388 12038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5f917016c3945c190d784dea7b4ce39: Generated new TSK 0
I20260810 11:49:48.754703 12038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:48.773936 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11946
W20260810 11:49:48.775151 12012 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.106:57769
W20260810 11:49:48.775947 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 11946: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:48.776131 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 11946
2026-08-10T11:49:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:48 dist-test-slave-njxd krb5kdc[12053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:48 dist-test-slave-njxd krb5kdc[12053](info): set up 2 sockets
Aug 10 11:49:48 dist-test-slave-njxd krb5kdc[12053](info): commencing operation
krb5kdc: starting...
W20260810 11:49:49.965610 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:50 dist-test-slave-njxd krb5kdc[12053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:50.313632 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43753
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:50.681723 12067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:50.682060 12067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:50.682229 12067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:50.689904 12067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:50.690065 12067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:50.690181 12067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:50.690285 12067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:50.690371 12067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:50.700996 12067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:50.702466 12067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:50.703907 12067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:50.716953 12072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:50.717053 12073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:50.717053 12075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:50.718055 12067 server_base.cc:1034] running on GCE node
I20260810 11:49:50.718981 12067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:50.720598 12067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:50.721813 12067 hybrid_clock.cc:648] HybridClock initialized: now 1786362590721752 us; error 56 us; skew 500 ppm
I20260810 11:49:50.722229 12067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:50 dist-test-slave-njxd krb5kdc[12053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362590, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:50.725796 12067 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:50.727594 12067 webserver.cc:460] Webserver started at http://127.20.93.190:36469/ using document root <none> and password file <none>
I20260810 11:49:50.728448 12067 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:50.728615 12067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:50.728960 12067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:50.732352 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "cd2e9394b7cd40deaa753c87613f5e56"
format_stamp: "Formatted at 2026-08-10 11:49:50 on dist-test-slave-njxd"
server_key: "a80e97fd5bf24b17fa90b50f5c40196f"
server_key_iv: "a0ca7ffc46e8bc006eb2a212375185c2"
server_key_version: "encryptionkey@0"
I20260810 11:49:50.733209 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "cd2e9394b7cd40deaa753c87613f5e56"
format_stamp: "Formatted at 2026-08-10 11:49:50 on dist-test-slave-njxd"
server_key: "a80e97fd5bf24b17fa90b50f5c40196f"
server_key_iv: "a0ca7ffc46e8bc006eb2a212375185c2"
server_key_version: "encryptionkey@0"
I20260810 11:49:50.739251 12067 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 11:49:50.743435 12082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:50.745056 12067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:49:50.745286 12067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "cd2e9394b7cd40deaa753c87613f5e56"
format_stamp: "Formatted at 2026-08-10 11:49:50 on dist-test-slave-njxd"
server_key: "a80e97fd5bf24b17fa90b50f5c40196f"
server_key_iv: "a0ca7ffc46e8bc006eb2a212375185c2"
server_key_version: "encryptionkey@0"
I20260810 11:49:50.745524 12067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:50.769448 12067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:50.773921 12067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:50.774370 12067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:50.794466 12067 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:43753
I20260810 11:49:50.794484 12138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:43753 every 8 connection(s)
I20260810 11:49:50.796056 12067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:50.799624 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12067
I20260810 11:49:50.799813 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:50.800318 20854 external_mini_cluster.cc:1428] Setting key 8224bdd771d8613dd0ba9f25766a3345
I20260810 11:49:50.800333 12139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:50.815167 12139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56: Bootstrap starting.
Aug 10 11:49:50 dist-test-slave-njxd krb5kdc[12053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:50.819211 12139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:50.820640 12139 log.cc:824] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:50.824338 12139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56: No bootstrap required, opened a new log
I20260810 11:49:50.826190 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:50.802307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40765 (local address 127.20.93.190:43753)
0810 11:49:50.803122 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:50.803142 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:50.803200 (+ 58us) server_negotiation.cc:407] Connection header received
0810 11:49:50.805483 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:50.805512 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:50.806090 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:50.806555 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:50.807868 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:50.809879 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:50.811228 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:50.812339 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:50.812534 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:50.816925 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:50.816937 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:50.816957 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:50.817022 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:50.821026 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:50.822223 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:50.822232 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:50.822243 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:50.822445 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:50.822942 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:50.822946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:50.822950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:50.823672 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:50.823989 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:50.824431 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 11:49:50.824773 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":338,"threads_started":1}
I20260810 11:49:50.830515 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:50.833565 12139 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } }
I20260810 11:49:50.834026 12139 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:50.834156 12139 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2e9394b7cd40deaa753c87613f5e56, State: Initialized, Role: FOLLOWER
I20260810 11:49:50.834890 12139 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } }
I20260810 11:49:50.835282 12139 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:50.835472 12139 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:50.835650 12139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:50.838534 12139 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } }
I20260810 11:49:50.839190 12139 leader_election.cc:302] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd2e9394b7cd40deaa753c87613f5e56; no voters:
I20260810 11:49:50.839900 12139 leader_election.cc:288] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:50.840016 12149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:50.842847 12139 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:50.844300 12149 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [term 1 LEADER]: Becoming Leader. State: Replica: cd2e9394b7cd40deaa753c87613f5e56, State: Running, Role: LEADER
I20260810 11:49:50.845324 12149 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } }
I20260810 11:49:50.849252 12152 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2e9394b7cd40deaa753c87613f5e56. Latest consensus state: current_term: 1 leader_uuid: "cd2e9394b7cd40deaa753c87613f5e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } } }
I20260810 11:49:50.849781 12152 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:50.850474 12150 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd2e9394b7cd40deaa753c87613f5e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2e9394b7cd40deaa753c87613f5e56" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43753 } } }
I20260810 11:49:50.850833 12150 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:50.854681 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:50.835067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53629 (local address 127.20.93.190:43753)
0810 11:49:50.835240 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:50.835246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:50.835912 (+ 666us) server_negotiation.cc:407] Connection header received
0810 11:49:50.836080 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:50.836087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:50.836182 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:50.836296 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:50.837779 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:50.838912 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:50.840448 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:50.841128 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:50.841245 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:50.844309 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:50.844316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:50.844321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:50.844373 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:50.848514 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:50.849916 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:50.849921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:50.849925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:50.850126 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:50.851191 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:50.851200 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:50.851206 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:50.851805 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:50.853545 (+ 1740us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:50.853810 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 11:49:50.854069 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 11:49:50.859850 12158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:50.865901 12158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:50.876022 12158 catalog_manager.cc:1347] Generated new cluster ID: 183b3a18b88d4d6eb3d0dc1bd1a041ef
I20260810 11:49:50.876183 12158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:50.915206 12158 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:50.916567 12158 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:50.922890 12158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd2e9394b7cd40deaa753c87613f5e56: Generated new TSK 0
I20260810 11:49:50.923808 12158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:50.937866 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12067
I20260810 11:49:50.939617 12134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:51.035429 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12067
W20260810 11:49:51.056344 12051 connection.cc:535] client connection to 127.20.93.190:43753 recv error: Network error: failed to read from TLS socket (remote: 127.20.93.190:43753): Connection reset by peer (error 104)
2026-08-10T11:49:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:51 dist-test-slave-njxd krb5kdc[12174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:51 dist-test-slave-njxd krb5kdc[12174](info): set up 2 sockets
Aug 10 11:49:51 dist-test-slave-njxd krb5kdc[12174](info): commencing operation
krb5kdc: starting...
W20260810 11:49:52.241901 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:52 dist-test-slave-njxd krb5kdc[12174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:52.595070 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43689
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:52.954955 12189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:52.955322 12189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:52.955487 12189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:52.963124 12189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:52.963290 12189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:52.963402 12189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:52.963517 12189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:52.963610 12189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:52.973861 12189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:52.975297 12189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:52.976696 12189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:52.988795 12194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:52.988827 12195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:52.989281 12197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:52.989691 12189 server_base.cc:1034] running on GCE node
I20260810 11:49:52.990454 12189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:52.991869 12189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:52.993076 12189 hybrid_clock.cc:648] HybridClock initialized: now 1786362592993029 us; error 40 us; skew 500 ppm
I20260810 11:49:52.993433 12189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:52 dist-test-slave-njxd krb5kdc[12174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362592, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:52.996703 12189 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:52.998157 12189 webserver.cc:460] Webserver started at http://127.20.93.190:34281/ using document root <none> and password file <none>
I20260810 11:49:52.998807 12189 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:52.998910 12189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:52.999161 12189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:53.002460 12189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "f7ccb438fb14498baa329c7e04c3b4d5"
format_stamp: "Formatted at 2026-08-10 11:49:52 on dist-test-slave-njxd"
server_key: "f3f192f21a1dd81c2a52e50bfd2ebebe"
server_key_iv: "8a307887209e254d63d18eb07aadeead"
server_key_version: "encryptionkey@0"
I20260810 11:49:53.003160 12189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "f7ccb438fb14498baa329c7e04c3b4d5"
format_stamp: "Formatted at 2026-08-10 11:49:52 on dist-test-slave-njxd"
server_key: "f3f192f21a1dd81c2a52e50bfd2ebebe"
server_key_iv: "8a307887209e254d63d18eb07aadeead"
server_key_version: "encryptionkey@0"
I20260810 11:49:53.008253 12189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:49:53.011690 12204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:53.013084 12189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:49:53.013227 12189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "f7ccb438fb14498baa329c7e04c3b4d5"
format_stamp: "Formatted at 2026-08-10 11:49:52 on dist-test-slave-njxd"
server_key: "f3f192f21a1dd81c2a52e50bfd2ebebe"
server_key_iv: "8a307887209e254d63d18eb07aadeead"
server_key_version: "encryptionkey@0"
I20260810 11:49:53.013391 12189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:53.035080 12189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:53.039736 12189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:53.039999 12189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:53.059386 12189 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:43689
I20260810 11:49:53.059381 12261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:43689 every 8 connection(s)
I20260810 11:49:53.060837 12189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:53.064786 12262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:53.069777 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12189
I20260810 11:49:53.069993 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:53.070626 20854 external_mini_cluster.cc:1428] Setting key d9dbb8d83037f2360078cf21d7049494
I20260810 11:49:53.079063 12262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5: Bootstrap starting.
I20260810 11:49:53.083781 12262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:53.085413 12262 log.cc:824] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:53.089114 12262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5: No bootstrap required, opened a new log
I20260810 11:49:53.093984 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:53.074564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56351 (local address 127.20.93.190:43689)
0810 11:49:53.077611 (+ 3047us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:53.077630 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:53.077704 (+ 74us) server_negotiation.cc:407] Connection header received
0810 11:49:53.080558 (+ 2854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:53.080594 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:53.081186 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:53.081729 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:53.082889 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:53.085092 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:53.086750 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:53.088071 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:53.088305 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:53.090467 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:53.090477 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:53.090497 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:53.091272 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:53.091384 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:53.091425 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:53.091607 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:53.092263 (+ 656us) server_negotiation.cc:299] Negotiation successful
0810 11:49:53.092484 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2751,"thread_start_us":2470,"threads_started":1}
I20260810 11:49:53.099175 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:53.101121 12262 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } }
I20260810 11:49:53.101824 12262 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:53.101948 12262 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ccb438fb14498baa329c7e04c3b4d5, State: Initialized, Role: FOLLOWER
I20260810 11:49:53.102623 12262 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } }
I20260810 11:49:53.102933 12262 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:53.103082 12262 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:53.103281 12262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:53.106213 12262 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } }
I20260810 11:49:53.106767 12262 leader_election.cc:302] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7ccb438fb14498baa329c7e04c3b4d5; no voters:
I20260810 11:49:53.107508 12262 leader_election.cc:288] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:53.107662 12272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:53.109406 12272 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [term 1 LEADER]: Becoming Leader. State: Replica: f7ccb438fb14498baa329c7e04c3b4d5, State: Running, Role: LEADER
I20260810 11:49:53.110172 12272 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } }
I20260810 11:49:53.110339 12262 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:53.112234 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:53.104650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44689 (local address 127.20.93.190:43689)
0810 11:49:53.104881 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:53.104894 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:53.105321 (+ 427us) server_negotiation.cc:407] Connection header received
0810 11:49:53.105528 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:53.105534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:53.105653 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:53.105719 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:53.106420 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:53.107363 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:53.109180 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:53.109873 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:53.110028 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:53.111357 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:53.111362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:53.111365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:53.111386 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:53.111426 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:53.111442 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:53.111577 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:53.111784 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 11:49:53.111841 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:49:53.114982 12273 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } } }
I20260810 11:49:53.115077 12274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7ccb438fb14498baa329c7e04c3b4d5. Latest consensus state: current_term: 1 leader_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ccb438fb14498baa329c7e04c3b4d5" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43689 } } }
I20260810 11:49:53.115530 12274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:53.116183 12273 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:53.116473 12279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:53.124958 12279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:53.135537 12279 catalog_manager.cc:1347] Generated new cluster ID: 62c6a953cc6e4a109535d66de37e665c
I20260810 11:49:53.135699 12279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:53.153841 12279 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:53.155603 12279 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:53.163455 12279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7ccb438fb14498baa329c7e04c3b4d5: Generated new TSK 0
I20260810 11:49:53.164337 12279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:53.183167 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12189
W20260810 11:49:53.185207 12257 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56351
W20260810 11:49:53.186064 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12189: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:53.186218 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12189
W20260810 11:49:53.208969 12172 connection.cc:535] client connection to 127.20.93.190:43689 recv error: Network error: failed to read from TLS socket (remote: 127.20.93.190:43689): Connection reset by peer (error 104)
2026-08-10T11:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:53 dist-test-slave-njxd krb5kdc[12295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:53 dist-test-slave-njxd krb5kdc[12295](info): set up 2 sockets
Aug 10 11:49:53 dist-test-slave-njxd krb5kdc[12295](info): commencing operation
krb5kdc: starting...
W20260810 11:49:54.389216 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:54 dist-test-slave-njxd krb5kdc[12295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:54.741987 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38471
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:38471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:55.097816 12309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:55.098114 12309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:55.098304 12309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:55.105779 12309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:55.106053 12309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:55.106124 12309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:55.106235 12309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:55.106312 12309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:55.116604 12309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:38471
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:38471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:55.118083 12309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:55.119467 12309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:55.131557 12316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:55.131634 12315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:55.131659 12318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:55.132828 12309 server_base.cc:1034] running on GCE node
I20260810 11:49:55.133790 12309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:55.135399 12309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:55.136613 12309 hybrid_clock.cc:648] HybridClock initialized: now 1786362595136575 us; error 26 us; skew 500 ppm
I20260810 11:49:55.137009 12309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:55 dist-test-slave-njxd krb5kdc[12295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362595, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:55.140484 12309 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:55.142258 12309 webserver.cc:460] Webserver started at http://127.20.93.190:41425/ using document root <none> and password file <none>
I20260810 11:49:55.143013 12309 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:55.143157 12309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:55.143510 12309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:55.146960 12309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "bfc421418b0e493c95f240c1ab5575e6"
format_stamp: "Formatted at 2026-08-10 11:49:55 on dist-test-slave-njxd"
server_key: "b8e89a623b32cceca3c3690402093ef2"
server_key_iv: "2cc9b46cae1103dd4470d7d2e43c2b83"
server_key_version: "encryptionkey@0"
I20260810 11:49:55.147814 12309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "bfc421418b0e493c95f240c1ab5575e6"
format_stamp: "Formatted at 2026-08-10 11:49:55 on dist-test-slave-njxd"
server_key: "b8e89a623b32cceca3c3690402093ef2"
server_key_iv: "2cc9b46cae1103dd4470d7d2e43c2b83"
server_key_version: "encryptionkey@0"
I20260810 11:49:55.153481 12309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:49:55.157320 12326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:55.158788 12309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:49:55.158990 12309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "bfc421418b0e493c95f240c1ab5575e6"
format_stamp: "Formatted at 2026-08-10 11:49:55 on dist-test-slave-njxd"
server_key: "b8e89a623b32cceca3c3690402093ef2"
server_key_iv: "2cc9b46cae1103dd4470d7d2e43c2b83"
server_key_version: "encryptionkey@0"
I20260810 11:49:55.159243 12309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:55.165655 12309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:55.169991 12309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:55.170332 12309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:55.190191 12309 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:38471
I20260810 11:49:55.190228 12385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:38471 every 8 connection(s)
I20260810 11:49:55.191752 12309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:55.195688 12386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:55.201033 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12309
I20260810 11:49:55.201251 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:55.201802 20854 external_mini_cluster.cc:1428] Setting key 92c2b0481118e6c689e9432e282314d8
I20260810 11:49:55.207420 12386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6: Bootstrap starting.
I20260810 11:49:55.211736 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:55.203688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53321 (local address 127.20.93.190:38471)
0810 11:49:55.204667 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:55.204680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:55.204729 (+ 49us) server_negotiation.cc:407] Connection header received
0810 11:49:55.206734 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:55.206758 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:55.207120 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:55.207532 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:55.209235 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:55.209264 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:55.209285 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:55.209305 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:55.209390 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:55.209425 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:55.209716 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:55.210315 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 11:49:55.210566 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":477,"threads_started":1}
I20260810 11:49:55.212041 12386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:55.213487 12386 log.cc:824] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:55.216194 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:55.216884 12386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6: No bootstrap required, opened a new log
I20260810 11:49:55.224288 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:55.220714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54145 (local address 127.20.93.190:38471)
0810 11:49:55.220888 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:55.220892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:55.222105 (+ 1213us) server_negotiation.cc:407] Connection header received
0810 11:49:55.222336 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:55.222340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:55.222394 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:55.222515 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:55.223517 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:55.223521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:55.223524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:55.223539 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:55.223580 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:55.223599 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:55.223772 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:55.223882 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 11:49:55.223948 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:49:55.226751 12386 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } }
I20260810 11:49:55.227186 12386 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:55.227312 12386 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc421418b0e493c95f240c1ab5575e6, State: Initialized, Role: FOLLOWER
I20260810 11:49:55.227897 12386 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } }
I20260810 11:49:55.228174 12386 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:55.228329 12386 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:55.228518 12386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:55.231091 12386 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } }
I20260810 11:49:55.231705 12386 leader_election.cc:302] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc421418b0e493c95f240c1ab5575e6; no voters:
I20260810 11:49:55.232365 12386 leader_election.cc:288] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:55.232570 12397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:55.233676 12397 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [term 1 LEADER]: Becoming Leader. State: Replica: bfc421418b0e493c95f240c1ab5575e6, State: Running, Role: LEADER
I20260810 11:49:55.234354 12397 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } }
I20260810 11:49:55.234894 12386 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:55.237214 12400 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc421418b0e493c95f240c1ab5575e6. Latest consensus state: current_term: 1 leader_uuid: "bfc421418b0e493c95f240c1ab5575e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } } }
I20260810 11:49:55.237717 12400 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:55.238309 12399 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc421418b0e493c95f240c1ab5575e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc421418b0e493c95f240c1ab5575e6" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 38471 } } }
I20260810 11:49:55.238693 12399 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:55.243928 12406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:55.249828 12406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:55.260370 12406 catalog_manager.cc:1347] Generated new cluster ID: 1423e40990784163b292354524acc9f3
I20260810 11:49:55.260532 12406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:55.272246 12406 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:55.272753 12406 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:55.281517 12406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc421418b0e493c95f240c1ab5575e6: Generated new TSK 0
I20260810 11:49:55.282536 12406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:55.288874 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12309
W20260810 11:49:55.290333 12381 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53321
W20260810 11:49:55.291172 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12309: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:55.291338 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12309
2026-08-10T11:49:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2104 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:55 dist-test-slave-njxd krb5kdc[12422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:55 dist-test-slave-njxd krb5kdc[12422](info): set up 2 sockets
Aug 10 11:49:55 dist-test-slave-njxd krb5kdc[12422](info): commencing operation
krb5kdc: starting...
W20260810 11:49:56.488758 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:56 dist-test-slave-njxd krb5kdc[12422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:56.841470 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:39599
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:39599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:57.219329 12437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:57.219651 12437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:57.219825 12437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:57.227427 12437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:57.227552 12437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:57.227685 12437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:57.227789 12437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:57.227856 12437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:57.238301 12437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:39599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:39599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:57.239728 12437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:57.241191 12437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:57.254014 12443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:57.253983 12444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:57.253933 12446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:57.255211 12437 server_base.cc:1034] running on GCE node
I20260810 11:49:57.256043 12437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:57.257511 12437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:57.258776 12437 hybrid_clock.cc:648] HybridClock initialized: now 1786362597258735 us; error 34 us; skew 500 ppm
I20260810 11:49:57.259157 12437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:57 dist-test-slave-njxd krb5kdc[12422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362597, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:57.267963 12437 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:57.269797 12437 webserver.cc:460] Webserver started at http://127.20.93.190:42331/ using document root <none> and password file <none>
I20260810 11:49:57.270560 12437 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:57.270675 12437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:57.270965 12437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:57.274371 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "db80f76348194466804f937797d3bf01"
format_stamp: "Formatted at 2026-08-10 11:49:57 on dist-test-slave-njxd"
server_key: "c379f97eda99c7d1d7c3c7cd87a2eb8e"
server_key_iv: "f82c15c9a8f835c5d5700a92ad689a3d"
server_key_version: "encryptionkey@0"
I20260810 11:49:57.275286 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "db80f76348194466804f937797d3bf01"
format_stamp: "Formatted at 2026-08-10 11:49:57 on dist-test-slave-njxd"
server_key: "c379f97eda99c7d1d7c3c7cd87a2eb8e"
server_key_iv: "f82c15c9a8f835c5d5700a92ad689a3d"
server_key_version: "encryptionkey@0"
I20260810 11:49:57.281057 12437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 11:49:57.284952 12453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:57.286492 12437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:49:57.286652 12437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "db80f76348194466804f937797d3bf01"
format_stamp: "Formatted at 2026-08-10 11:49:57 on dist-test-slave-njxd"
server_key: "c379f97eda99c7d1d7c3c7cd87a2eb8e"
server_key_iv: "f82c15c9a8f835c5d5700a92ad689a3d"
server_key_version: "encryptionkey@0"
I20260810 11:49:57.286854 12437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:57.311164 12437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:57.315868 12437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:57.316179 12437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:57.335955 12437 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:39599
I20260810 11:49:57.335991 12509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:39599 every 8 connection(s)
I20260810 11:49:57.337764 12437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:57.338199 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12437
I20260810 11:49:57.338388 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:57.339077 20854 external_mini_cluster.cc:1428] Setting key e953d354f0b3edfbfde9ede7ad88c1a4
I20260810 11:49:57.342610 12510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:49:57 dist-test-slave-njxd krb5kdc[12422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:49:57.356675 12510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01: Bootstrap starting.
I20260810 11:49:57.360240 12510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:57.361346 12510 log.cc:824] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:57.363617 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:57.340975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58403 (local address 127.20.93.190:39599)
0810 11:49:57.341851 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:57.341863 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:57.341909 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:49:57.343673 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:57.343702 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:57.344095 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:57.344438 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:57.346421 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:57.347834 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:57.349798 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:57.350516 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:57.350655 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:57.355791 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:57.355800 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:57.355817 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:57.355863 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:57.359173 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:57.359957 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:57.359964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:57.359971 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:57.360146 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:57.360662 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:57.360666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:57.360668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:57.361242 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:57.361569 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:57.362126 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 11:49:57.362601 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":390,"threads_started":1}
I20260810 11:49:57.364645 12510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01: No bootstrap required, opened a new log
I20260810 11:49:57.367528 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:57.375059 12510 raft_consensus.cc:348] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } }
I20260810 11:49:57.375443 12510 raft_consensus.cc:374] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:57.375578 12510 raft_consensus.cc:729] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db80f76348194466804f937797d3bf01, State: Initialized, Role: FOLLOWER
I20260810 11:49:57.376350 12510 consensus_queue.cc:260] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } }
I20260810 11:49:57.376628 12510 raft_consensus.cc:388] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:57.376767 12510 raft_consensus.cc:482] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:57.376978 12510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:57.379963 12510 raft_consensus.cc:504] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } }
I20260810 11:49:57.380498 12510 leader_election.cc:302] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db80f76348194466804f937797d3bf01; no voters:
I20260810 11:49:57.381143 12510 leader_election.cc:288] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:57.381321 12524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:57.383394 12510 sys_catalog.cc:564] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:57.383358 12524 raft_consensus.cc:686] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [term 1 LEADER]: Becoming Leader. State: Replica: db80f76348194466804f937797d3bf01, State: Running, Role: LEADER
I20260810 11:49:57.384012 12524 consensus_queue.cc:237] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } }
I20260810 11:49:57.387485 12526 sys_catalog.cc:455] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db80f76348194466804f937797d3bf01. Latest consensus state: current_term: 1 leader_uuid: "db80f76348194466804f937797d3bf01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } } }
I20260810 11:49:57.387926 12526 sys_catalog.cc:458] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:57.388451 12525 sys_catalog.cc:455] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db80f76348194466804f937797d3bf01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db80f76348194466804f937797d3bf01" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 39599 } } }
I20260810 11:49:57.388710 12525 sys_catalog.cc:458] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:57.390784 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:57.373179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38553 (local address 127.20.93.190:39599)
0810 11:49:57.373400 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:57.373412 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:57.373865 (+ 453us) server_negotiation.cc:407] Connection header received
0810 11:49:57.374019 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:57.374025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:57.374112 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:57.374222 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:57.375754 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:57.376857 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:57.378215 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:57.378926 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:57.379067 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:57.382002 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:57.382011 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:57.382019 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:49:57.382075 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:57.386278 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:57.387317 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:57.387324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:57.387329 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:57.387569 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:49:57.388102 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:49:57.388109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:49:57.388117 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:49:57.388572 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:57.388876 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:57.389865 (+ 989us) server_negotiation.cc:299] Negotiation successful
0810 11:49:57.390174 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:49:57.404088 12534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:57.409384 12534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:57.418551 12534 catalog_manager.cc:1347] Generated new cluster ID: d6257019df2f413b823b14b6e7a8f19e
I20260810 11:49:57.418712 12534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:57.430383 12534 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:57.431846 12534 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:57.438681 12534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db80f76348194466804f937797d3bf01: Generated new TSK 0
I20260810 11:49:57.439589 12534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:57.454444 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12437
I20260810 11:49:57.455686 12503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:49:57.549058 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12437
2026-08-10T11:49:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2258 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:57 dist-test-slave-njxd krb5kdc[12547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:57 dist-test-slave-njxd krb5kdc[12547](info): set up 2 sockets
Aug 10 11:49:57 dist-test-slave-njxd krb5kdc[12547](info): commencing operation
krb5kdc: starting...
W20260810 11:49:58.774369 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:49:59 dist-test-slave-njxd krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:49:59.149847 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35793
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:35793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:49:59.538081 12562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:49:59.538420 12562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:49:59.538520 12562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:49:59.546671 12562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:49:59.546818 12562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:49:59.546866 12562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:49:59.546923 12562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:49:59.546969 12562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:49:59.557492 12562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:35793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:49:59.558930 12562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:49:59.560297 12562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:49:59.573352 12567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:59.573298 12570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:49:59.573271 12568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:49:59.574244 12562 server_base.cc:1034] running on GCE node
I20260810 11:49:59.575239 12562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:49:59.576910 12562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:49:59.578146 12562 hybrid_clock.cc:648] HybridClock initialized: now 1786362599578083 us; error 64 us; skew 500 ppm
I20260810 11:49:59.578540 12562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:49:59 dist-test-slave-njxd krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:49:59.581912 12562 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:49:59.583555 12562 webserver.cc:460] Webserver started at http://127.20.93.190:37547/ using document root <none> and password file <none>
I20260810 11:49:59.584295 12562 fs_manager.cc:359] Metadata directory not provided
I20260810 11:49:59.584455 12562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:49:59.584822 12562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:49:59.588053 12562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "a43f4738658e4e95875e821a0f8cea03"
format_stamp: "Formatted at 2026-08-10 11:49:59 on dist-test-slave-njxd"
server_key: "ffc12c8f331f92b8524d4d62746d70b5"
server_key_iv: "bdfa52b13e472f5c2aa2fa6b5329f779"
server_key_version: "encryptionkey@0"
I20260810 11:49:59.588868 12562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "a43f4738658e4e95875e821a0f8cea03"
format_stamp: "Formatted at 2026-08-10 11:49:59 on dist-test-slave-njxd"
server_key: "ffc12c8f331f92b8524d4d62746d70b5"
server_key_iv: "bdfa52b13e472f5c2aa2fa6b5329f779"
server_key_version: "encryptionkey@0"
I20260810 11:49:59.594502 12562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:49:59.598331 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:49:59.599750 12562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:49:59.599961 12562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "a43f4738658e4e95875e821a0f8cea03"
format_stamp: "Formatted at 2026-08-10 11:49:59 on dist-test-slave-njxd"
server_key: "ffc12c8f331f92b8524d4d62746d70b5"
server_key_iv: "bdfa52b13e472f5c2aa2fa6b5329f779"
server_key_version: "encryptionkey@0"
I20260810 11:49:59.600174 12562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:49:59.614588 12562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:49:59.619450 12562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:49:59.619797 12562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:49:59.640939 12562 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:35793
I20260810 11:49:59.640964 12632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:35793 every 8 connection(s)
I20260810 11:49:59.642645 12562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:49:59.646899 12633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:49:59.648185 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12562
I20260810 11:49:59.648386 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:49:59.648983 20854 external_mini_cluster.cc:1428] Setting key d5eb06a51935b892786767485e475a9f
I20260810 11:49:59.662859 12633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03: Bootstrap starting.
I20260810 11:49:59.665899 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:59.651544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41507 (local address 127.20.93.190:35793)
0810 11:49:59.652737 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:59.652761 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:59.653249 (+ 488us) server_negotiation.cc:407] Connection header received
0810 11:49:59.655166 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:59.655198 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:59.655813 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:59.656267 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:59.656792 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:59.658794 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:59.660212 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:59.661307 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:59.661512 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:59.662851 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:59.662863 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:59.662884 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:59.663645 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:59.663748 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:59.663782 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:59.663907 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:59.664183 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 11:49:59.664464 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":530,"threads_started":1}
I20260810 11:49:59.666970 12633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03: Neither blocks nor log segments found. Creating new log.
I20260810 11:49:59.668120 12633 log.cc:824] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03: Log is configured to *not* fsync() on all Append() calls
I20260810 11:49:59.670003 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:49:59.671730 12633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03: No bootstrap required, opened a new log
I20260810 11:49:59.682274 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:49:59.675346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57495 (local address 127.20.93.190:35793)
0810 11:49:59.675534 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:49:59.675540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:49:59.676103 (+ 563us) server_negotiation.cc:407] Connection header received
0810 11:49:59.676248 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:49:59.676254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:49:59.676339 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:49:59.676453 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:49:59.676958 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:59.678191 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:59.679537 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:49:59.680295 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:49:59.680438 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:49:59.681489 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:49:59.681496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:49:59.681500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:49:59.681523 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:49:59.681569 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:49:59.681613 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:49:59.681703 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 11:49:59.681846 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 11:49:59.681944 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:49:59.682389 12633 raft_consensus.cc:348] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } }
I20260810 11:49:59.682816 12633 raft_consensus.cc:374] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:49:59.682945 12633 raft_consensus.cc:729] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43f4738658e4e95875e821a0f8cea03, State: Initialized, Role: FOLLOWER
I20260810 11:49:59.683542 12633 consensus_queue.cc:260] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } }
I20260810 11:49:59.683805 12633 raft_consensus.cc:388] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:49:59.683974 12633 raft_consensus.cc:482] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:49:59.684154 12633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:49:59.686542 12633 raft_consensus.cc:504] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } }
I20260810 11:49:59.687067 12633 leader_election.cc:302] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a43f4738658e4e95875e821a0f8cea03; no voters:
I20260810 11:49:59.687728 12633 leader_election.cc:288] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:49:59.687913 12644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:49:59.688675 12644 raft_consensus.cc:686] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [term 1 LEADER]: Becoming Leader. State: Replica: a43f4738658e4e95875e821a0f8cea03, State: Running, Role: LEADER
I20260810 11:49:59.689309 12644 consensus_queue.cc:237] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } }
I20260810 11:49:59.689951 12633 sys_catalog.cc:564] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:49:59.693128 12645 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a43f4738658e4e95875e821a0f8cea03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } } }
I20260810 11:49:59.693442 12647 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a43f4738658e4e95875e821a0f8cea03. Latest consensus state: current_term: 1 leader_uuid: "a43f4738658e4e95875e821a0f8cea03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43f4738658e4e95875e821a0f8cea03" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 35793 } } }
I20260810 11:49:59.693723 12645 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:59.693817 12647 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03 [sys.catalog]: This master's current role is: LEADER
I20260810 11:49:59.696578 12653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:49:59.702039 12653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:49:59.711673 12653 catalog_manager.cc:1347] Generated new cluster ID: 696066b631fd4770821dfa0bfbdad5a2
I20260810 11:49:59.711795 12653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:49:59.722108 12653 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:49:59.723515 12653 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:49:59.734072 12653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a43f4738658e4e95875e821a0f8cea03: Generated new TSK 0
I20260810 11:49:59.734957 12653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:49:59.749071 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12562
W20260810 11:49:59.750636 12628 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41507
W20260810 11:49:59.751387 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:49:59.751542 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12562
2026-08-10T11:49:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:49:59 dist-test-slave-njxd krb5kdc[12670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:49:59 dist-test-slave-njxd krb5kdc[12670](info): set up 2 sockets
Aug 10 11:49:59 dist-test-slave-njxd krb5kdc[12670](info): commencing operation
krb5kdc: starting...
W20260810 11:50:00.960858 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:50:01 dist-test-slave-njxd krb5kdc[12670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:50:01.329684 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:45025
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:34361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:50:01.697289 12684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:01.697656 12684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:01.697845 12684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:01.705366 12684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:01.705497 12684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:01.705662 12684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:01.705775 12684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:01.705862 12684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:01.716229 12684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:34361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:45025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:01.717993 12684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:01.719498 12684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:01.732851 12690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:01.732918 12691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:01.732964 12693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:01.734315 12684 server_base.cc:1034] running on GCE node
I20260810 11:50:01.735196 12684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:01.736842 12684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:01.738067 12684 hybrid_clock.cc:648] HybridClock initialized: now 1786362601738014 us; error 45 us; skew 500 ppm
I20260810 11:50:01.738441 12684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:01 dist-test-slave-njxd krb5kdc[12670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:01.742110 12684 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:50:01.743794 12684 webserver.cc:460] Webserver started at http://127.20.93.190:44531/ using document root <none> and password file <none>
I20260810 11:50:01.744592 12684 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:01.744719 12684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:01.745007 12684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:01.748375 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "cc741db9413045af9f703726697e1077"
format_stamp: "Formatted at 2026-08-10 11:50:01 on dist-test-slave-njxd"
server_key: "902cd854fe0b5be45ad9e25b25300721"
server_key_iv: "e3a91b90c49f92f9c5f269f89723f160"
server_key_version: "encryptionkey@0"
I20260810 11:50:01.749241 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "cc741db9413045af9f703726697e1077"
format_stamp: "Formatted at 2026-08-10 11:50:01 on dist-test-slave-njxd"
server_key: "902cd854fe0b5be45ad9e25b25300721"
server_key_iv: "e3a91b90c49f92f9c5f269f89723f160"
server_key_version: "encryptionkey@0"
I20260810 11:50:01.755363 12684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 11:50:01.759487 12702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:01.761209 12684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:50:01.761456 12684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "cc741db9413045af9f703726697e1077"
format_stamp: "Formatted at 2026-08-10 11:50:01 on dist-test-slave-njxd"
server_key: "902cd854fe0b5be45ad9e25b25300721"
server_key_iv: "e3a91b90c49f92f9c5f269f89723f160"
server_key_version: "encryptionkey@0"
I20260810 11:50:01.761783 12684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:01.768651 12684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:01.773943 12684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:01.774340 12684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:01.796726 12684 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:45025
I20260810 11:50:01.796746 12760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:45025 every 8 connection(s)
I20260810 11:50:01.798442 12684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:50:01.802728 12761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:01.805514 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12684
I20260810 11:50:01.805727 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:50:01.806288 20854 external_mini_cluster.cc:1428] Setting key ba06f27ed42171ce70f3c8710f1a2d0b
I20260810 11:50:01.815212 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:01.808358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51091 (local address 127.20.93.190:45025)
0810 11:50:01.809326 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:01.809340 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:01.809387 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:50:01.811300 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:01.811326 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:01.811750 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:01.812145 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:01.813338 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:01.813348 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:01.813367 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:50:01.813390 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:01.813455 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:50:01.813479 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:01.813774 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:01.813893 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 11:50:01.814166 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":466,"threads_started":1}
I20260810 11:50:01.815867 12761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077: Bootstrap starting.
I20260810 11:50:01.819309 12761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:01.819409 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:50:01.820643 12761 log.cc:824] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077: Log is configured to *not* fsync() on all Append() calls
I20260810 11:50:01.824596 12761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077: No bootstrap required, opened a new log
I20260810 11:50:01.827955 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:01.824982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33943 (local address 127.20.93.190:45025)
0810 11:50:01.825243 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:01.825251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:01.825783 (+ 532us) server_negotiation.cc:407] Connection header received
0810 11:50:01.826047 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:01.826052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:01.826126 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:01.826230 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:01.827236 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:01.827241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:01.827244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:50:01.827259 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:01.827305 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:50:01.827321 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:01.827424 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:01.827566 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 11:50:01.827616 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:50:01.834324 12761 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } }
I20260810 11:50:01.834774 12761 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:50:01.834909 12761 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc741db9413045af9f703726697e1077, State: Initialized, Role: FOLLOWER
I20260810 11:50:01.835525 12761 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } }
I20260810 11:50:01.835796 12761 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:50:01.835949 12761 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:50:01.836133 12761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:01.838999 12761 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } }
I20260810 11:50:01.839529 12761 leader_election.cc:302] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc741db9413045af9f703726697e1077; no voters:
I20260810 11:50:01.840152 12761 leader_election.cc:288] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:50:01.840298 12771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:50:01.841079 12771 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [term 1 LEADER]: Becoming Leader. State: Replica: cc741db9413045af9f703726697e1077, State: Running, Role: LEADER
I20260810 11:50:01.841710 12771 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } }
I20260810 11:50:01.842479 12761 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:01.844512 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc741db9413045af9f703726697e1077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } } }
I20260810 11:50:01.844940 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:01.848824 12777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:50:01.848781 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc741db9413045af9f703726697e1077. Latest consensus state: current_term: 1 leader_uuid: "cc741db9413045af9f703726697e1077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc741db9413045af9f703726697e1077" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 45025 } } }
I20260810 11:50:01.849093 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:01.854728 12777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:50:01.865097 12777 catalog_manager.cc:1347] Generated new cluster ID: e1f94445ec704fcbb815ceef576f88ff
I20260810 11:50:01.865247 12777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:50:01.879611 12777 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:50:01.880174 12777 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:50:01.887678 12777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc741db9413045af9f703726697e1077: Generated new TSK 0
I20260810 11:50:01.888711 12777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:50:01.895706 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12684
W20260810 11:50:01.897022 12755 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51091
W20260810 11:50:01.898061 20854 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12684: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:50:01.898206 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12684
2026-08-10T11:50:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2154 ms)
[----------] 18 tests from AuthTokenIssuingTest (39284 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:50:02 dist-test-slave-njxd krb5kdc[12794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:50:02 dist-test-slave-njxd krb5kdc[12794](info): set up 2 sockets
Aug 10 11:50:02 dist-test-slave-njxd krb5kdc[12794](info): commencing operation
krb5kdc: starting...
W20260810 11:50:03.108273 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.005s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:50:03 dist-test-slave-njxd krb5kdc[12794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:50:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:50:03.463783 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43449
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:50:03.824011 12810 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:03.824389 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:03.824515 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:03.824667 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:03.832664 12810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:03.832765 12810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:03.832867 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:03.832964 12810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:03.833060 12810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:03.843601 12810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:43449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:03.845084 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:03.846594 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:03.858614 12816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:03.858777 12818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:03.858856 12820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:03.859761 12810 server_base.cc:1034] running on GCE node
I20260810 11:50:03.860657 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:03.862236 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:03.863494 12810 hybrid_clock.cc:648] HybridClock initialized: now 1786362603863425 us; error 50 us; skew 500 ppm
I20260810 11:50:03.863849 12810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:03 dist-test-slave-njxd krb5kdc[12794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:03.872848 12810 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:50:03.874598 12810 webserver.cc:460] Webserver started at http://127.20.93.190:38369/ using document root <none> and password file <none>
I20260810 11:50:03.875367 12810 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:03.875515 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:03.875856 12810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:03.879232 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "98fa78f4f27242dbb0003b66c86e0807"
format_stamp: "Formatted at 2026-08-10 11:50:03 on dist-test-slave-njxd"
server_key: "8835481000b72ef1d487a4315c88572b"
server_key_iv: "5ed64a1ff078968b59983315423a152a"
server_key_version: "encryptionkey@0"
I20260810 11:50:03.880133 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "98fa78f4f27242dbb0003b66c86e0807"
format_stamp: "Formatted at 2026-08-10 11:50:03 on dist-test-slave-njxd"
server_key: "8835481000b72ef1d487a4315c88572b"
server_key_iv: "5ed64a1ff078968b59983315423a152a"
server_key_version: "encryptionkey@0"
I20260810 11:50:03.885819 12810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 11:50:03.889792 12828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:03.891296 12810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:50:03.891512 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "98fa78f4f27242dbb0003b66c86e0807"
format_stamp: "Formatted at 2026-08-10 11:50:03 on dist-test-slave-njxd"
server_key: "8835481000b72ef1d487a4315c88572b"
server_key_iv: "5ed64a1ff078968b59983315423a152a"
server_key_version: "encryptionkey@0"
I20260810 11:50:03.891760 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:03.909217 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:03.913782 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:03.914077 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:03.934167 12810 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:43449
I20260810 11:50:03.934207 12885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:43449 every 8 connection(s)
I20260810 11:50:03.935611 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:50:03.939347 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12810
I20260810 11:50:03.939569 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:50:03.939812 12887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:03.940296 20854 external_mini_cluster.cc:1428] Setting key a21f623a2a9d04dbfead8e1b76a27d01
I20260810 11:50:03.946808 12887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:03 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:03.973765 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:03.951976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.20.93.190:43449)
0810 11:50:03.952831 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:03.952840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:03.952899 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:50:03.954403 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:03.954413 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:03.954843 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:03.955090 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:03.956405 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:03.958447 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:03.959939 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:03.960964 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:03.961187 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:03.964788 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:03.964800 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:03.964821 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:03.964866 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:03.969046 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:03.969986 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:03.969997 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:03.970003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:03.970217 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:03.970624 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:03.970632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:03.970638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:03.971443 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:03.971664 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:03.972308 (+ 644us) server_negotiation.cc:299] Negotiation successful
0810 11:50:03.972805 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":632,"thread_start_us":466,"threads_started":1}
I20260810 11:50:03.973803 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:03.950506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43449 (local address 127.0.0.1:41852)
0810 11:50:03.951495 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:03.951537 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:03.953982 (+ 2445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:03.955148 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:03.955154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:03.955785 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:03.956162 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:03.956182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:03.958634 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:03.958638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:03.959772 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:03.959787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:03.961206 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:03.961211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:03.961365 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:03.962175 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:03.962200 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:03.964506 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:03.969317 (+ 4811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:03.969328 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:03.969338 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:03.969844 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:03.970333 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:03.970337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:03.970339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:03.970491 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:03.971587 (+ 1096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:03.971594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:03.971810 (+ 216us) client_negotiation.cc:769] Sending connection context
0810 11:50:03.972088 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 11:50:03.972580 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":408,"threads_started":1}
WARNING: no policy specified for kudu/127.20.93.189@KRBTEST.COM; defaulting to no policy
I20260810 11:50:03.977083 12887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:03.978384 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:03.977974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54752)
0810 11:50:03.978141 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:03.978264 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:50:03.978822 12832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.189@KRBTEST.COM" created.
I20260810 11:50:03.980841 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:03.980315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54768)
0810 11:50:03.980510 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:03.980625 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 11:50:03.981099 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:44399: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
I20260810 11:50:04.021526 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:44399
--webserver_interface=127.20.93.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:50:04.030508 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } attempt: 1
I20260810 11:50:04.031903 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.031305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54770)
0810 11:50:04.031552 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.031699 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 11:50:04.033185 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.032716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54782)
0810 11:50:04.032899 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.032996 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:50:04.033435 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:44399: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
I20260810 11:50:04.101778 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } attempt: 2
I20260810 11:50:04.103014 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.102526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54786)
0810 11:50:04.102773 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.102874 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 11:50:04.104032 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.103656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54796)
0810 11:50:04.103804 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.103903 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:50:04.104210 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:44399: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
I20260810 11:50:04.195499 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } attempt: 3
I20260810 11:50:04.196691 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.196242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54812)
0810 11:50:04.196454 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.196552 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 11:50:04.197791 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.197346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54824)
0810 11:50:04.197504 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.197628 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 11:50:04.197983 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:44399: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
I20260810 11:50:04.341336 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } attempt: 4
I20260810 11:50:04.342588 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.342059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54832)
0810 11:50:04.342365 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.342456 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260810 11:50:04.343600 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.343255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54842)
0810 11:50:04.343398 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.343484 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 11:50:04.343798 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:44399: Network error: Client connection negotiation failed: client connection to 127.20.93.189:44399: connect: Connection refused (error 111)
W20260810 11:50:04.375589 12895 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:04.376015 12895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:04.376152 12895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:04.376318 12895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:04.383641 12895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:04.383780 12895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:04.383888 12895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:04.383978 12895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:04.384070 12895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:04.394753 12895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:44399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.93.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:04.396263 12895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:04.397701 12895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:04.410218 12900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:04.410192 12902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:04.410255 12904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:04.411103 12895 server_base.cc:1034] running on GCE node
I20260810 11:50:04.411943 12895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:04.413443 12895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:04.414665 12895 hybrid_clock.cc:648] HybridClock initialized: now 1786362604414580 us; error 61 us; skew 500 ppm
I20260810 11:50:04.415032 12895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:04 dist-test-slave-njxd krb5kdc[12794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:04.418524 12895 init.cc:377] Logged in from keytab as kudu/127.20.93.189@KRBTEST.COM (short username kudu)
I20260810 11:50:04.420099 12895 webserver.cc:460] Webserver started at http://127.20.93.189:40037/ using document root <none> and password file <none>
I20260810 11:50:04.420856 12895 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:04.420995 12895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:04.421331 12895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:04.424548 12895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data/instance:
uuid: "8fce2337e6944e1d98af532c1c0a8035"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "8fb3ed75a2810c4d2708063bbde50cc7"
server_key_iv: "04ac16e41f15f0632259aad4d73a45c7"
server_key_version: "encryptionkey@0"
I20260810 11:50:04.425412 12895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal/instance:
uuid: "8fce2337e6944e1d98af532c1c0a8035"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "8fb3ed75a2810c4d2708063bbde50cc7"
server_key_iv: "04ac16e41f15f0632259aad4d73a45c7"
server_key_version: "encryptionkey@0"
I20260810 11:50:04.431149 12895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:50:04.434914 12911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:04.436394 12895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:50:04.436596 12895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "8fce2337e6944e1d98af532c1c0a8035"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "8fb3ed75a2810c4d2708063bbde50cc7"
server_key_iv: "04ac16e41f15f0632259aad4d73a45c7"
server_key_version: "encryptionkey@0"
I20260810 11:50:04.436844 12895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:04.456926 12895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:04.461562 12895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:04.461918 12895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:04.481371 12895 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.189:44399
I20260810 11:50:04.481392 12969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.189:44399 every 8 connection(s)
I20260810 11:50:04.482883 12895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
I20260810 11:50:04.486902 12970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:04.487219 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12895
I20260810 11:50:04.487416 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-1/wal/instance
I20260810 11:50:04.487861 20854 external_mini_cluster.cc:1428] Setting key a599c75f88ab26670d222c1197cf26ed
I20260810 11:50:04.493845 12970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:04 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:04.519330 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.496606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.20.93.190:43449)
0810 11:50:04.497151 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:04.497155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:04.498261 (+ 1106us) server_negotiation.cc:407] Connection header received
0810 11:50:04.500290 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:04.500296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:04.500388 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:04.500510 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:04.502365 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.503072 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.505549 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.506019 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.506119 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.511110 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:04.511118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:04.511123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:04.511190 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:04.515307 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.516345 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.516353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.516359 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.516523 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.517111 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.517119 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.517124 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.517562 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:04.517753 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:04.518519 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 11:50:04.518791 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":278,"threads_started":1}
I20260810 11:50:04.519964 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.497240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43449 (local address 127.0.0.1:41858)
0810 11:50:04.498094 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.498134 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:04.499973 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:04.500756 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:04.500764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:04.501298 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:04.502120 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.502147 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.503240 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.503248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.505397 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.505415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.506263 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.506272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.506591 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.507455 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:04.507489 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:04.510746 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:04.515486 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.515496 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.515517 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.516163 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.516668 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.516675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.516682 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.516970 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.517719 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:04.517732 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:04.518108 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 11:50:04.518367 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 11:50:04.518778 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":401,"threads_started":1}
I20260810 11:50:04.521848 12970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.93.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.188@KRBTEST.COM" created.
Aug 10 11:50:04 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:04.538393 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.522752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54844)
0810 11:50:04.522959 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.522985 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:04.523121 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:04.524102 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:04.524106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:04.524572 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:04.524699 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.524707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.526189 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.526197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.527043 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.527053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.527904 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.527910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.528101 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.528827 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:04.528855 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:04.531546 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:04.535971 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.535975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.535978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.536358 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.536779 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.536782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.536784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.536906 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.537686 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:04.537690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:04.537807 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 11:50:04.537917 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 11:50:04.538130 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 11:50:04.538890 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.522772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.20.93.189:44399)
0810 11:50:04.523693 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:04.523699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:04.523732 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:50:04.523774 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:04.523780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:04.523957 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:04.524114 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:04.524882 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.526057 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.527185 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.527748 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.527882 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.531794 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:04.531818 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:04.531840 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:04.531884 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:04.535825 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.536500 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.536511 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.536518 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.536668 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.537017 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.537024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.537030 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.537529 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:04.537802 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:04.538110 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 11:50:04.538406 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":600,"threads_started":1}
I20260810 11:50:04.541949 12970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:04.543323 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.542835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45840)
0810 11:50:04.543050 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.543171 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 11:50:04.543669 12915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.545275 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.544913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45842)
0810 11:50:04.545095 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.545174 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 11:50:04.545720 12970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
I20260810 11:50:04.570924 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:41497
--webserver_interface=127.20.93.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:50:04.595096 12970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 1
I20260810 11:50:04.596536 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.595894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45858)
0810 11:50:04.596159 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.596306 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 11:50:04.597878 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.597407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45864)
0810 11:50:04.597632 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.597748 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 11:50:04.598225 12970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.643042 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } attempt: 5
Aug 10 11:50:04 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:04.659161 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.643822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54854)
0810 11:50:04.644024 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.644048 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:04.644172 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:04.644561 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:04.644565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:04.644980 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:04.645171 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.645182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.646659 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.646664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.647550 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:04.647563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.648620 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.648625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:04.648820 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.649523 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:04.649542 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:04.652018 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:04.656426 (+ 4408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.656432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.656435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.656907 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.657388 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:04.657392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:04.657394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:04.657539 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:04.658252 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:04.658257 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:04.658478 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 11:50:04.658629 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 11:50:04.658904 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:50:04.659343 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.643937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.20.93.189:44399)
0810 11:50:04.644145 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:04.644151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:04.644177 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:50:04.644282 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:04.644287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:04.644391 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:04.644522 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:04.645375 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.646514 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.647733 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:04.648456 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:04.648600 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:04.652284 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:04.652291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:04.652296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:04.652343 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:04.656186 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.657063 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.657069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.657079 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.657259 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:04.657700 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:04.657707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:04.657710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:04.658118 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:04.658277 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:04.658777 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 11:50:04.659041 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:50:04.660545 12887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:04.661845 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.661343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45872)
0810 11:50:04.661533 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.661686 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 11:50:04.662997 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.662653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45882)
0810 11:50:04.662802 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.662876 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 11:50:04.663395 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.666452 12970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 2
I20260810 11:50:04.667646 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.667211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45890)
0810 11:50:04.667397 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.667490 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:50:04.668788 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.668360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45906)
0810 11:50:04.668512 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.668651 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 11:50:04.668999 12970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.714654 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 1
I20260810 11:50:04.716049 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.715514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45910)
0810 11:50:04.715761 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.715865 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 11:50:04.717168 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.716813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45924)
0810 11:50:04.716970 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.717053 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:50:04.717412 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.760285 12970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 3
I20260810 11:50:04.761518 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.761008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45938)
0810 11:50:04.761263 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.761361 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 11:50:04.762650 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.762225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45944)
0810 11:50:04.762463 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.762547 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260810 11:50:04.762876 12970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.785691 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 2
I20260810 11:50:04.786904 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.786397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45948)
0810 11:50:04.786622 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.786736 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 11:50:04.787887 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.787541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45960)
0810 11:50:04.787701 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.787786 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 11:50:04.788087 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.894318 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 3
I20260810 11:50:04.895591 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.895126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45962)
0810 11:50:04.895329 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.895423 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 11:50:04.896694 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.896295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45974)
0810 11:50:04.896518 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.896594 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260810 11:50:04.896888 12887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
I20260810 11:50:04.906078 12970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 4
I20260810 11:50:04.907212 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.906732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45984)
0810 11:50:04.906965 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.907050 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 11:50:04.908232 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:04.907887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45986)
0810 11:50:04.908037 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:04.908111 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:50:04.908435 12970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:41497: Network error: Client connection negotiation failed: client connection to 127.20.93.188:41497: connect: Connection refused (error 111)
W20260810 11:50:04.940796 12981 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:04.941255 12981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:04.941366 12981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:04.941505 12981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:04.950700 12981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:04.950798 12981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:04.950908 12981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:04.950990 12981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:04.951102 12981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:04.961377 12981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:43449,127.20.93.189:44399,127.20.93.188:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.93.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:04.962813 12981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:04.964285 12981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:04.976830 12987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:04.976951 12988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:04.977144 12990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:04.978107 12981 server_base.cc:1034] running on GCE node
I20260810 11:50:04.978976 12981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:04.980554 12981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:04.981755 12981 hybrid_clock.cc:648] HybridClock initialized: now 1786362604981662 us; error 59 us; skew 500 ppm
I20260810 11:50:04.982139 12981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:04 dist-test-slave-njxd krb5kdc[12794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:04.985526 12981 init.cc:377] Logged in from keytab as kudu/127.20.93.188@KRBTEST.COM (short username kudu)
I20260810 11:50:04.987149 12981 webserver.cc:460] Webserver started at http://127.20.93.188:37717/ using document root <none> and password file <none>
I20260810 11:50:04.987828 12981 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:04.987918 12981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:04.988222 12981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:04.991364 12981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data/instance:
uuid: "93310c9cd2204a55a24a04868e9b3bb3"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "05107d217018d81206e28a519c96115d"
server_key_iv: "1fb436d1c8d11d5bab9fc61c384aa0ae"
server_key_version: "encryptionkey@0"
I20260810 11:50:04.992146 12981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal/instance:
uuid: "93310c9cd2204a55a24a04868e9b3bb3"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "05107d217018d81206e28a519c96115d"
server_key_iv: "1fb436d1c8d11d5bab9fc61c384aa0ae"
server_key_version: "encryptionkey@0"
I20260810 11:50:04.998008 12981 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:50:05.001693 12997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:05.003177 12981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:50:05.003334 12981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "93310c9cd2204a55a24a04868e9b3bb3"
format_stamp: "Formatted at 2026-08-10 11:50:04 on dist-test-slave-njxd"
server_key: "05107d217018d81206e28a519c96115d"
server_key_iv: "1fb436d1c8d11d5bab9fc61c384aa0ae"
server_key_version: "encryptionkey@0"
I20260810 11:50:05.003538 12981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:05.019101 12981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:05.024088 12981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:05.024348 12981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:05.044243 12981 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.188:41497
I20260810 11:50:05.044267 13050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.188:41497 every 8 connection(s)
I20260810 11:50:05.045825 12981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
I20260810 11:50:05.049860 13051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:05.056285 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 12981
I20260810 11:50:05.056499 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786362228785499-20854-0/minicluster-data/master-2/wal/instance
I20260810 11:50:05.057054 20854 external_mini_cluster.cc:1428] Setting key 2f3a570b5a32f2382cc8a07bb6bc3b77
I20260810 11:50:05.056993 13051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:05.074150 12887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 4
I20260810 11:50:05.082449 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.060517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.20.93.190:43449)
0810 11:50:05.061833 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.061840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.061862 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:50:05.062749 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.062757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.062856 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.062977 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.064720 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.065469 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.067057 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.067446 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.067601 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.072385 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.072393 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.072398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.072475 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.076921 (+ 4446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.078083 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.078089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.078093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.078318 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.078973 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.078977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.078980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.079393 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.079570 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.081799 (+ 2229us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.082029 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1188,"thread_start_us":1070,"threads_started":1}
I20260810 11:50:05.082883 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.059161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.20.93.188:41497)
0810 11:50:05.059954 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.059968 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.060018 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:50:05.062462 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.062485 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.062798 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.063019 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.064728 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.065995 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.067257 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.067831 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.067973 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.073456 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.073470 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.073490 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.073553 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.078289 (+ 4736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.079767 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.079777 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.079780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.079978 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.080539 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.080544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.080547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.081080 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.081410 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.082124 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.082424 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":378,"threads_started":1}
I20260810 11:50:05.082480 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.060318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:43449 (local address 127.0.0.1:41864)
0810 11:50:05.061258 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:05.061328 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:05.062473 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:05.063513 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:05.063518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:05.064113 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:05.064525 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.064545 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.065714 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.065718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.066885 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.066897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.067688 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.067694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.067920 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.068698 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:05.068720 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:05.072050 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:05.077108 (+ 5058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.077121 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.077136 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.077906 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.078465 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.078470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.078476 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.078813 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.079645 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:05.079654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:05.080180 (+ 526us) client_negotiation.cc:769] Sending connection context
0810 11:50:05.080512 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 11:50:05.081073 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":349,"threads_started":1}
I20260810 11:50:05.086299 13051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:05.094671 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.076371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:45998)
0810 11:50:05.076720 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:05.076742 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:05.076864 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:05.079691 (+ 2827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:05.079697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:05.080433 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:05.080717 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.080730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.082501 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.082509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.083727 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.083741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.084596 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.084603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.084845 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.085555 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:05.085622 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:05.088517 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:05.091905 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.091911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.091916 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.092327 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.092789 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.092795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.092800 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.092990 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.093542 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:05.093547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:05.093766 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 11:50:05.093939 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 11:50:05.094258 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260810 11:50:05.095677 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.077109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.20.93.188:41497)
0810 11:50:05.079216 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.079224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.079246 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:50:05.079322 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.079327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.079436 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.080616 (+ 1180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.080914 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.082300 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.083887 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.084405 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.084554 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.088759 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.088766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.088771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.088807 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.091771 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.092451 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.092458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.092462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.092643 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.093113 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.093120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.093123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.093402 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.093562 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.095000 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.095253 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1874,"thread_start_us":1565,"threads_started":1}
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:05.103138 12887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: Bootstrap starting.
I20260810 11:50:05.103823 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.086958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:44399 (local address 127.0.0.1:54862)
0810 11:50:05.087147 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:05.087165 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:05.087264 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:05.088058 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:05.088063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:05.088538 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:05.088760 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.088769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.091192 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.091197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.092673 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.092686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.093844 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.093850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.094055 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.094658 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:05.094683 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:05.097398 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:05.100871 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.100876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.100881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.101280 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.101795 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.101802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.101807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.102021 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.102751 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:05.102756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:05.102936 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 11:50:05.103119 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 11:50:05.103436 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 11:50:05.103855 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.087343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.20.93.189:44399)
0810 11:50:05.087603 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.087610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.087629 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:50:05.087689 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.087694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.087776 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.087878 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.088911 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.089713 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.092891 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.093650 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.093791 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.097625 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.097632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.097637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.097686 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.100705 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.101425 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.101432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.101436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.101644 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.102175 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.102180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.102183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.102584 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.102751 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.103217 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.103438 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:50:05.104852 13051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:05.105899 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.086446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.20.93.190:43449)
0810 11:50:05.086717 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.086725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.086746 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:50:05.086852 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.086864 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.086961 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.087067 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.088576 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.089704 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.091127 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.091780 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.091906 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.097989 (+ 6083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.097998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.098006 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.098052 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.101717 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.102597 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.102602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.102606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.102790 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.103296 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.103301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.103304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.103706 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.103999 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.105221 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.105469 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260810 11:50:05.109254 12887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:05.110775 12887 log.cc:824] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:05.114758 12887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: No bootstrap required, opened a new log
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:05.121791 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.105918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.20.93.188:41497)
0810 11:50:05.106295 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.106299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.106486 (+ 187us) server_negotiation.cc:407] Connection header received
0810 11:50:05.106673 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.106680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.106778 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.106881 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.107895 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.108950 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.110680 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.111260 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.111418 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.115503 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.115512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.115521 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.115551 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.118680 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.119544 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.119548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.119552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.119685 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.120062 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.120067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.120071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.120462 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.120664 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.121089 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.121339 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
I20260810 11:50:05.121922 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.105844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:46012)
0810 11:50:05.106374 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:05.106396 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:05.106521 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:05.107041 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:05.107046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:05.107535 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:05.107741 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.107753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.109104 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.109111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.110490 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.110510 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.111625 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.111631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.111894 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.112612 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:05.112637 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:05.115300 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:05.118858 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.118867 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.118872 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.119396 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.119799 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.119804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.119808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.119948 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.120710 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:05.120713 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:05.120833 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 11:50:05.120980 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 11:50:05.121503 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":175}
I20260810 11:50:05.126279 12887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.126732 12887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98fa78f4f27242dbb0003b66c86e0807, State: Initialized, Role: FOLLOWER
I20260810 11:50:05.127306 12887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.127565 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.108106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.20.93.189:44399)
0810 11:50:05.108317 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.108321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.108336 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:50:05.108377 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.108380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.108431 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.108552 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.109956 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.110798 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.112531 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.113301 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.113444 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.119639 (+ 6195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.119647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.119652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.119699 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.123496 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.124489 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.124495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.124499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.124686 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.125396 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.125402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.125405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.125862 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.126168 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.126913 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.127203 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 11:50:05.128188 13051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: Bootstrap starting.
I20260810 11:50:05.129081 13059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.129796 13059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.130937 12887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:05.132586 13051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:05.133553 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:50:05.134213 13051 log.cc:824] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: Log is configured to *not* fsync() on all Append() calls
I20260810 11:50:05.137146 12859 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "98fa78f4f27242dbb0003b66c86e0807"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41866
I20260810 11:50:05.137696 12859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 11:50:05.137935 12859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:05.138464 13051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: No bootstrap required, opened a new log
I20260810 11:50:05.142354 12859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.145397 12859 leader_election.cc:288] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [CANDIDATE]: Term 1 election: Requested vote from peers 8fce2337e6944e1d98af532c1c0a8035 (127.20.93.189:44399), 93310c9cd2204a55a24a04868e9b3bb3 (127.20.93.188:41497)
I20260810 11:50:05.150442 13051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.150949 13051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93310c9cd2204a55a24a04868e9b3bb3, State: Initialized, Role: FOLLOWER
I20260810 11:50:05.151562 13051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.153107 13074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.154227 13074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.154441 13051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:05.154531 13025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98fa78f4f27242dbb0003b66c86e0807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "93310c9cd2204a55a24a04868e9b3bb3"
I20260810 11:50:05.155126 13025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:05.156176 13073 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:50:05.159824 13025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98fa78f4f27242dbb0003b66c86e0807 in term 1.
I20260810 11:50:05.160682 12833 leader_election.cc:302] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93310c9cd2204a55a24a04868e9b3bb3, 98fa78f4f27242dbb0003b66c86e0807; no voters:
I20260810 11:50:05.161324 13059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:50:05.165007 13059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [term 1 LEADER]: Becoming Leader. State: Replica: 98fa78f4f27242dbb0003b66c86e0807, State: Running, Role: LEADER
I20260810 11:50:05.165879 13059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
W20260810 11:50:05.167168 12835 tablet.cc:2369] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:50:05.177740 13092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:50:05.178013 13092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:50:05.162268 12810 master_runner.cc:186] Error getting master registration for 127.20.93.189:44399: OK, instance_id { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" instance_seqno: 1786362604475732 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:50:05.186573 12810 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:50:05.186628 13078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98fa78f4f27242dbb0003b66c86e0807. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.187114 13078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:05.188094 13096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:50:05.200712 13096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:50:05.207614 12970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } attempt: 5
I20260810 11:50:05.211269 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.168806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.20.93.190:43449)
0810 11:50:05.168984 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.168991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.181832 (+ 12841us) server_negotiation.cc:407] Connection header received
0810 11:50:05.182298 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.182305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.182394 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.182476 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.186612 (+ 4136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.187950 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.191546 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.192270 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.192382 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.198592 (+ 6210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.198600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.198605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.198648 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.202843 (+ 4195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.204359 (+ 1516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.204365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.204368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.204536 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.205687 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.205692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.205695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.206094 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.206402 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.207462 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.210568 (+ 3106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 11:50:05.204314 12981 master_runner.cc:186] Error getting master registration for 127.20.93.189:44399: OK, instance_id { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" instance_seqno: 1786362604475732 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:50:05.215435 12981 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 11:50:05 dist-test-slave-njxd krb5kdc[12794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:05.228108 13025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [term 1 FOLLOWER]: Refusing update from remote peer 98fa78f4f27242dbb0003b66c86e0807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:50:05.228698 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.174201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.20.93.189:44399)
0810 11:50:05.174390 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.174397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.183189 (+ 8792us) server_negotiation.cc:407] Connection header received
0810 11:50:05.186671 (+ 3482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.186679 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.186769 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.186844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.194150 (+ 7306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.195650 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.199095 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.199751 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.199855 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.209030 (+ 9175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.209038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.209043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.209097 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.216903 (+ 7806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.221658 (+ 4755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.221668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.221674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.221881 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.222488 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.222496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.222502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.222896 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.223130 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.225363 (+ 2233us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.225645 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:50:05.229256 13078 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:50:05.232296 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.208328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:41497 (local address 127.0.0.1:46030)
0810 11:50:05.208687 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:05.208707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:05.208818 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:05.209786 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:05.209794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:05.211327 (+ 1533us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:05.211539 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.211551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.213081 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.213087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.214705 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:05.214727 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.217079 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.217085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:05.217320 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.217942 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:05.217966 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:05.220733 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:05.227305 (+ 6572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.227322 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.227326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.227879 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.228811 (+ 932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:05.228816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:05.228839 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:05.229107 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:05.230291 (+ 1184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:05.230297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:05.230556 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 11:50:05.230798 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 11:50:05.231249 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260810 11:50:05.234408 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.209074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.20.93.188:41497)
0810 11:50:05.209242 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.209249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.209269 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:50:05.209328 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.209332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.209411 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.209508 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.211727 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.212891 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.214878 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.215577 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.215714 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.220999 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.221006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.221012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.221053 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.227065 (+ 6012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.228133 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.228142 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.228146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.228600 (+ 454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.229376 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.229381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.229384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.230071 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.233170 (+ 3099us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.233419 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.233862 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:50:05.236743 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.180421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.20.93.188:41497)
0810 11:50:05.180593 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.180599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.190532 (+ 9933us) server_negotiation.cc:407] Connection header received
0810 11:50:05.191160 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.191170 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.191279 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.191372 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.197818 (+ 6446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.198892 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.204841 (+ 5949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.205687 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.205797 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.223693 (+ 17896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:05.223700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:05.223708 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:05.223753 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:05.228198 (+ 4445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.232916 (+ 4718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.232922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.232926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.233128 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:05.233913 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:05.233919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:05.233922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:05.234284 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:05.234594 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.235658 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.236063 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:50:05.241155 12970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: Bootstrap starting.
I20260810 11:50:05.249954 12970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:05.252184 12970 log.cc:824] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: Log is configured to *not* fsync() on all Append() calls
I20260810 11:50:05.256572 12970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: No bootstrap required, opened a new log
I20260810 11:50:05.262952 13074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98fa78f4f27242dbb0003b66c86e0807. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.263293 13074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.264082 13098 mvcc.cc:205] Tried to move back new op lower bound from 7316941231001821184 to 7316941230765142016. Current Snapshot: MvccSnapshot[applied={T|T < 7316941231001821184}]
I20260810 11:50:05.267242 13074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.267521 13074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.268000 13078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.268290 13078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:05.268586 13099 mvcc.cc:205] Tried to move back new op lower bound from 7316941231001821184 to 7316941230765142016. Current Snapshot: MvccSnapshot[applied={T|T < 7316941231001821184}]
I20260810 11:50:05.269025 12970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.270265 12970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fce2337e6944e1d98af532c1c0a8035, State: Initialized, Role: FOLLOWER
I20260810 11:50:05.271077 12970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } }
I20260810 11:50:05.286237 12970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:05.287809 13096 catalog_manager.cc:1347] Generated new cluster ID: ad010cc1f9f74e10ad5575111c76804a
I20260810 11:50:05.287931 13096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:50:05.285634 13105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.302794 13105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.297870 12941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98fa78f4f27242dbb0003b66c86e0807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8fce2337e6944e1d98af532c1c0a8035"
I20260810 11:50:05.303915 12940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:05.308056 12940 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [term 1 FOLLOWER]: Refusing update from remote peer 98fa78f4f27242dbb0003b66c86e0807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:50:05.310832 13059 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:50:05.316102 12941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98fa78f4f27242dbb0003b66c86e0807 in term 1.
I20260810 11:50:05.332724 13096 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:50:05.334836 13096 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:50:05.349701 13096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807: Generated new TSK 0
I20260810 11:50:05.350847 13096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:50:05.395987 13105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98fa78f4f27242dbb0003b66c86e0807. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.396386 13105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.396883 13113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.397372 13113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:05.398082 13078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98fa78f4f27242dbb0003b66c86e0807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fa78f4f27242dbb0003b66c86e0807" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 43449 } } peers { permanent_uuid: "8fce2337e6944e1d98af532c1c0a8035" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 44399 } } peers { permanent_uuid: "93310c9cd2204a55a24a04868e9b3bb3" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 41497 } } }
I20260810 11:50:05.398375 13078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fa78f4f27242dbb0003b66c86e0807 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:05.443890 13123 catalog_manager.cc:1259] Loaded cluster ID: ad010cc1f9f74e10ad5575111c76804a
I20260810 11:50:05.449671 13123 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: loading cluster ID for follower catalog manager: success
I20260810 11:50:05.453923 13123 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: acquiring CA information for follower catalog manager: success
I20260810 11:50:05.460369 13123 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fce2337e6944e1d98af532c1c0a8035: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 11:50:05.521960 13056 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.509658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.20.93.190:43449)
0810 11:50:05.509934 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.509940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.518738 (+ 8798us) server_negotiation.cc:407] Connection header received
0810 11:50:05.520149 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.520155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.520247 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.520330 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.521593 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41876: BlockingRecv error: recv got EOF from 127.0.0.1:41876 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260810 11:50:05.526690 12976 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.509165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.20.93.189:44399)
0810 11:50:05.509419 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.509425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.523671 (+ 14246us) server_negotiation.cc:407] Connection header received
0810 11:50:05.524636 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.524642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.524740 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.524820 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.526336 (+ 1516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54878: BlockingRecv error: recv got EOF from 127.0.0.1:54878 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 11:50:05.530911 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.513429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.20.93.188:41497)
0810 11:50:05.513631 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.513637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.524284 (+ 10647us) server_negotiation.cc:407] Connection header received
0810 11:50:05.524450 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.524456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.524556 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.524679 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.530615 (+ 5936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46034: BlockingRecv error: recv got EOF from 127.0.0.1:46034 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:50:05.563874 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.543862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.20.93.189:44399)
0810 11:50:05.544071 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.544079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.555426 (+ 11347us) server_negotiation.cc:407] Connection header received
0810 11:50:05.555727 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.555733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.555981 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.556165 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.556956 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.558147 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.560316 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.561100 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.561220 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.562164 (+ 944us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.562951 (+ 787us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.563087 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.563356 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.563475 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 11:50:05.612648 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.599050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.20.93.188:41497)
0810 11:50:05.599299 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.599306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.610638 (+ 11332us) server_negotiation.cc:407] Connection header received
0810 11:50:05.610943 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.610949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.611091 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.611210 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.612373 (+ 1163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46050: BlockingRecv error: recv got EOF from 127.0.0.1:46050 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:50:05.667948 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.650203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54900 (local address 127.20.93.189:44399)
0810 11:50:05.650402 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.650409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.660778 (+ 10369us) server_negotiation.cc:407] Connection header received
0810 11:50:05.661310 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.661317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.661435 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.661500 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.662260 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.663439 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.665483 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.666198 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.666304 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.666995 (+ 691us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.667164 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.667230 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.667532 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.667641 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 11:50:05.724440 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.701431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.20.93.188:41497)
0810 11:50:05.701680 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.701688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.722237 (+ 20549us) server_negotiation.cc:407] Connection header received
0810 11:50:05.722495 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.722503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.722630 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.722752 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.724151 (+ 1399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46066: BlockingRecv error: recv got EOF from 127.0.0.1:46066 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:50:05.785404 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.770065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.20.93.189:44399)
0810 11:50:05.770276 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.770283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.777928 (+ 7645us) server_negotiation.cc:407] Connection header received
0810 11:50:05.778507 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.778514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.778630 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.778707 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.779506 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.780699 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.782817 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.783528 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.783660 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.784348 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.784599 (+ 251us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.784679 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.784944 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.785045 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 11:50:05.829526 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.820060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.20.93.188:41497)
0810 11:50:05.820302 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.820309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.827342 (+ 7033us) server_negotiation.cc:407] Connection header received
0810 11:50:05.827904 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.827911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.828045 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.828142 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.829268 (+ 1126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46078: BlockingRecv error: recv got EOF from 127.0.0.1:46078 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:50:05.860390 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.851204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.20.93.189:44399)
0810 11:50:05.851914 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.851921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.852988 (+ 1067us) server_negotiation.cc:407] Connection header received
0810 11:50:05.853321 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.853327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.853430 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.853499 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.854599 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.855835 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.858045 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.858781 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.858927 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.859509 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.859669 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.859787 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.859973 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.860070 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560}
W20260810 11:50:05.871196 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.868592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.20.93.188:41497)
0810 11:50:05.868857 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.868864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.869465 (+ 601us) server_negotiation.cc:407] Connection header received
0810 11:50:05.869705 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.869713 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.869825 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.869947 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.870952 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46082: BlockingRecv error: recv got EOF from 127.0.0.1:46082 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
I20260810 11:50:05.890151 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.882746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.20.93.189:44399)
0810 11:50:05.882970 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.882976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.883550 (+ 574us) server_negotiation.cc:407] Connection header received
0810 11:50:05.883789 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.883795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.883890 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.884012 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.884541 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.885766 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.887818 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.888555 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.888708 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.889199 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.889346 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.889450 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.889643 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.889732 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:50:05.906528 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.903340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.20.93.188:41497)
0810 11:50:05.903574 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.903581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.904184 (+ 603us) server_negotiation.cc:407] Connection header received
0810 11:50:05.904524 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.904534 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.904642 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.904727 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.906278 (+ 1551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46094: BlockingRecv error: recv got EOF from 127.0.0.1:46094 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:50:05.934787 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.926973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.20.93.189:44399)
0810 11:50:05.927175 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.927181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.927700 (+ 519us) server_negotiation.cc:407] Connection header received
0810 11:50:05.928033 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.928040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.928138 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.928215 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.928959 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.930266 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.932248 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.933003 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.933125 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.933808 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.933946 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.934009 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.934231 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.934320 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 11:50:05.951582 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.949179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.20.93.188:41497)
0810 11:50:05.949381 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.949389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.950182 (+ 793us) server_negotiation.cc:407] Connection header received
0810 11:50:05.950403 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.950407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.950482 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.950590 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:05.951351 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46098: BlockingRecv error: recv got EOF from 127.0.0.1:46098 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 11:50:05.992821 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:05.987335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.20.93.189:44399)
0810 11:50:05.987654 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.987661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:05.987929 (+ 268us) server_negotiation.cc:407] Connection header received
0810 11:50:05.988181 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:05.988186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:05.988279 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:05.988384 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:05.988822 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.989687 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.991054 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:05.991522 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:05.991665 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:05.992053 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:05.992160 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:05.992260 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:05.992412 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 11:50:05.992499 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 11:50:06.001667 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:05.999155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46114 (local address 127.20.93.188:41497)
0810 11:50:05.999360 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:05.999366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.000218 (+ 852us) server_negotiation.cc:407] Connection header received
0810 11:50:06.000385 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.000391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.000509 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.000646 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:06.001390 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46114: BlockingRecv error: recv got EOF from 127.0.0.1:46114 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:50:06.074839 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.069194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.20.93.189:44399)
0810 11:50:06.069452 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.069462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.069789 (+ 327us) server_negotiation.cc:407] Connection header received
0810 11:50:06.070053 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.070057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.070135 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.070247 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.070720 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.071496 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.072888 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.073336 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.073499 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.074026 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.074159 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.074267 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.074429 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.074540 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 11:50:06.082940 13057 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:06.081118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.20.93.188:41497)
0810 11:50:06.081319 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.081324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.081660 (+ 336us) server_negotiation.cc:407] Connection header received
0810 11:50:06.081854 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.081858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.081937 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.082033 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:06.082760 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46116: BlockingRecv error: recv got EOF from 127.0.0.1:46116 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:50:06.180261 13092 catalog_manager.cc:1259] Loaded cluster ID: ad010cc1f9f74e10ad5575111c76804a
I20260810 11:50:06.180379 13092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: loading cluster ID for follower catalog manager: success
I20260810 11:50:06.183029 13092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: acquiring CA information for follower catalog manager: success
I20260810 11:50:06.185406 13092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 93310c9cd2204a55a24a04868e9b3bb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:50:06.220491 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.214770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.20.93.189:44399)
0810 11:50:06.215071 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.215078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.215407 (+ 329us) server_negotiation.cc:407] Connection header received
0810 11:50:06.215670 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.215674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.215750 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.215859 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.216341 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.217112 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.218660 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.219107 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.219255 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.219703 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.219839 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.219945 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.220115 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.220207 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 11:50:06.234861 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.227859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.20.93.188:41497)
0810 11:50:06.228132 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.228142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.228513 (+ 371us) server_negotiation.cc:407] Connection header received
0810 11:50:06.228783 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.228788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.228964 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.229100 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.229650 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.230434 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.232130 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.232614 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.232766 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.233425 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.234134 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.234308 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.234391 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.234492 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:50:06.260079 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.243628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.20.93.189:44399)
0810 11:50:06.244110 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.244119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.251080 (+ 6961us) server_negotiation.cc:407] Connection header received
0810 11:50:06.251533 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.251542 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.251685 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.251834 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.252915 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.254564 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.257930 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.258649 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.258801 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.259283 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.259395 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.259505 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.259690 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.259792 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310}
I20260810 11:50:06.264983 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.243349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.20.93.188:41497)
0810 11:50:06.243900 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.243910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.257008 (+ 13098us) server_negotiation.cc:407] Connection header received
0810 11:50:06.257269 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.257277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.257399 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.257501 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.258470 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.259385 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.262906 (+ 3521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.263380 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.263497 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.264095 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.264249 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.264349 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.264571 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.264655 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260810 11:50:06.266355 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:06.242939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.20.93.190:43449)
0810 11:50:06.256232 (+ 13293us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:06.256242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:06.258070 (+ 1828us) server_negotiation.cc:407] Connection header received
0810 11:50:06.258346 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:06.258374 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:06.258628 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:06.258819 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:06.260255 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.261034 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.263144 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:06.263696 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:06.263831 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:06.264525 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:06.264988 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:06.265096 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:06.265345 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:50:06.265446 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12919,"thread_start_us":13015,"threads_started":1}
I20260810 11:50:06.272048 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12810
I20260810 11:50:06.273336 12881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:06.386291 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12810
W20260810 11:50:06.411850 12914 connection.cc:535] server connection from 127.0.0.1:54854 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 11:50:06.412478 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12895
I20260810 11:50:06.413755 12965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:06.523491 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12895
I20260810 11:50:06.547571 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 12981
I20260810 11:50:06.548837 13046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:06.658317 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 12981
2026-08-10T11:50:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4756 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:50:06 dist-test-slave-njxd krb5kdc[13252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:50:06 dist-test-slave-njxd krb5kdc[13252](info): set up 2 sockets
Aug 10 11:50:06 dist-test-slave-njxd krb5kdc[13252](info): commencing operation
krb5kdc: starting...
W20260810 11:50:07.867393 20854 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/test-user.keytab.
Aug 10 11:50:08 dist-test-slave-njxd krb5kdc[13252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:50:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:50:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.93.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
Entry for principal kudu/127.20.93.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab.
I20260810 11:50:08.221837 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44319
--webserver_interface=127.20.93.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:50:08.582358 13266 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:08.582770 13266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:08.582904 13266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:08.583051 13266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:08.590646 13266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:08.590797 13266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:08.590909 13266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:08.591002 13266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:08.591101 13266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:08.601554 13266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.190.keytab
--principal=kudu/127.20.93.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.190:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.93.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:08.603075 13266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:08.604492 13266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:08.617282 13276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:08.617309 13272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:08.617439 13274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:08.618103 13266 server_base.cc:1034] running on GCE node
I20260810 11:50:08.618999 13266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:08.620543 13266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:08.621763 13266 hybrid_clock.cc:648] HybridClock initialized: now 1786362608621697 us; error 32 us; skew 500 ppm
I20260810 11:50:08.622138 13266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:08 dist-test-slave-njxd krb5kdc[13252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:08.625448 13266 init.cc:377] Logged in from keytab as kudu/127.20.93.190@KRBTEST.COM (short username kudu)
I20260810 11:50:08.626966 13266 webserver.cc:460] Webserver started at http://127.20.93.190:43271/ using document root <none> and password file <none>
I20260810 11:50:08.627707 13266 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:08.627822 13266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:08.628177 13266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:08.631322 13266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data/instance:
uuid: "5e00779a47e34a6ebac7685441be3074"
format_stamp: "Formatted at 2026-08-10 11:50:08 on dist-test-slave-njxd"
server_key: "17bbae38263d4c12bc18b63846156aac"
server_key_iv: "8e90c2799571736240481524b0d1e45b"
server_key_version: "encryptionkey@0"
I20260810 11:50:08.632169 13266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance:
uuid: "5e00779a47e34a6ebac7685441be3074"
format_stamp: "Formatted at 2026-08-10 11:50:08 on dist-test-slave-njxd"
server_key: "17bbae38263d4c12bc18b63846156aac"
server_key_iv: "8e90c2799571736240481524b0d1e45b"
server_key_version: "encryptionkey@0"
I20260810 11:50:08.638016 13266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:50:08.642107 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:08.643654 13266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:50:08.643874 13266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal
uuid: "5e00779a47e34a6ebac7685441be3074"
format_stamp: "Formatted at 2026-08-10 11:50:08 on dist-test-slave-njxd"
server_key: "17bbae38263d4c12bc18b63846156aac"
server_key_iv: "8e90c2799571736240481524b0d1e45b"
server_key_version: "encryptionkey@0"
I20260810 11:50:08.644105 13266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:08.666335 13266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:08.670825 13266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:08.671136 13266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:08.690896 13266 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.190:44319
I20260810 11:50:08.690914 13339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.190:44319 every 8 connection(s)
I20260810 11:50:08.692348 13266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/data/info.pb
I20260810 11:50:08.696103 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:08.698077 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 13266
I20260810 11:50:08.698252 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-0/wal/instance
I20260810 11:50:08.698753 20854 external_mini_cluster.cc:1428] Setting key 3d9184120c17663896329c126c3f4086
I20260810 11:50:08.703035 13340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:08 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:08.732818 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.707412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.20.93.190:44319)
0810 11:50:08.714096 (+ 6684us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:08.714106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:08.714141 (+ 35us) server_negotiation.cc:407] Connection header received
0810 11:50:08.714328 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:08.714339 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:08.714648 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:08.714821 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:08.716609 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:08.717884 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:08.720259 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:08.720856 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:08.721006 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:08.725350 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:08.725358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:08.725372 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:08.725400 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:08.728223 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:08.728910 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:08.728926 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:08.728928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:08.729105 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:08.729504 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:08.729512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:08.729515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:08.730093 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:08.730309 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:08.730876 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 11:50:08.731230 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6478,"thread_start_us":6347,"threads_started":1}
I20260810 11:50:08.732838 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.706812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44319 (local address 127.0.0.1:43258)
0810 11:50:08.707698 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.707756 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:08.710396 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:08.714927 (+ 4531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:08.714939 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:08.715849 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:08.716380 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:08.716408 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:08.718058 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:08.718067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:08.720021 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:08.720048 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:08.721158 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:08.721171 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:08.721401 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:08.722154 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:08.722188 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:08.725097 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:08.728356 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:08.728370 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:08.728382 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:08.728779 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:08.729237 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:08.729241 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:08.729245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:08.729399 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:08.730250 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:08.730258 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:08.730490 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 11:50:08.730748 (+ 258us) client_negotiation.cc:240] Negotiation successful
0810 11:50:08.731358 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":145,"thread_start_us":706,"threads_started":1}
WARNING: no policy specified for kudu/127.20.93.189@KRBTEST.COM; defaulting to no policy
I20260810 11:50:08.736181 13340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:08.737282 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.736858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53508)
0810 11:50:08.737018 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.737130 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.189@KRBTEST.COM" created.
W20260810 11:50:08.737638 13289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
I20260810 11:50:08.739495 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.739111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53514)
0810 11:50:08.739281 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.739352 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:50:08.739722 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:43705: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
Entry for principal kudu/127.20.93.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab.
I20260810 11:50:08.778214 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:43705
--webserver_interface=127.20.93.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:50:08.789108 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } attempt: 1
I20260810 11:50:08.790349 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.789877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53518)
0810 11:50:08.790080 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.790203 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 11:50:08.791415 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.791039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53534)
0810 11:50:08.791194 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.791275 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:50:08.791618 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:43705: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
I20260810 11:50:08.859917 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } attempt: 2
I20260810 11:50:08.861315 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.860686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53548)
0810 11:50:08.860968 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.861085 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:50:08.862708 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.862260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53558)
0810 11:50:08.862470 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.862564 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 11:50:08.862939 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:43705: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
I20260810 11:50:08.954298 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } attempt: 3
I20260810 11:50:08.955603 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.955133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53564)
0810 11:50:08.955351 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.955454 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 11:50:08.956749 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:08.956341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53574)
0810 11:50:08.956509 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:08.956592 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:50:08.957022 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:43705: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
I20260810 11:50:09.100317 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } attempt: 4
I20260810 11:50:09.101961 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.101205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53578)
0810 11:50:09.101686 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.101808 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260810 11:50:09.103058 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.102675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53588)
0810 11:50:09.102825 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.102927 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:50:09.103260 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.189:43705: Network error: Client connection negotiation failed: client connection to 127.20.93.189:43705: connect: Connection refused (error 111)
W20260810 11:50:09.138382 13347 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:09.138797 13347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:09.138931 13347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:09.139072 13347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:09.147380 13347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:09.147518 13347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:09.147626 13347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:09.147722 13347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:09.147812 13347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:09.158231 13347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.189.keytab
--principal=kudu/127.20.93.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.189:43705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.93.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:09.159657 13347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:09.161048 13347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:09.173195 13356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:09.173247 13359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:09.173285 13357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:09.174861 13347 server_base.cc:1034] running on GCE node
I20260810 11:50:09.175700 13347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:09.177206 13347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:09.178494 13347 hybrid_clock.cc:648] HybridClock initialized: now 1786362609178428 us; error 36 us; skew 500 ppm
I20260810 11:50:09.178932 13347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:09.187458 13347 init.cc:377] Logged in from keytab as kudu/127.20.93.189@KRBTEST.COM (short username kudu)
I20260810 11:50:09.189039 13347 webserver.cc:460] Webserver started at http://127.20.93.189:42531/ using document root <none> and password file <none>
I20260810 11:50:09.189868 13347 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:09.189970 13347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:09.190248 13347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:09.193344 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data/instance:
uuid: "f34313b57c4d4c36851a5012a95d77b0"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "7ce937554e0b6d4d6a896d49ff1e9335"
server_key_iv: "2f333cf009e48933bb8dfc6b0503464e"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.194255 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal/instance:
uuid: "f34313b57c4d4c36851a5012a95d77b0"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "7ce937554e0b6d4d6a896d49ff1e9335"
server_key_iv: "2f333cf009e48933bb8dfc6b0503464e"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.199857 13347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:50:09.203616 13368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:09.205013 13347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:50:09.205214 13347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal
uuid: "f34313b57c4d4c36851a5012a95d77b0"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "7ce937554e0b6d4d6a896d49ff1e9335"
server_key_iv: "2f333cf009e48933bb8dfc6b0503464e"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.205443 13347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:09.221172 13347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:09.225679 13347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:09.225972 13347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:09.245487 13347 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.189:43705
I20260810 11:50:09.245543 13422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.189:43705 every 8 connection(s)
I20260810 11:50:09.247098 13347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/data/info.pb
I20260810 11:50:09.251364 13423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:09.254040 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 13347
I20260810 11:50:09.254228 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-1/wal/instance
I20260810 11:50:09.254930 20854 external_mini_cluster.cc:1428] Setting key 56c31d7f6421476740a34763d534b91f
I20260810 11:50:09.258131 13423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:09.283910 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.261498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.20.93.190:44319)
0810 11:50:09.262763 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.262794 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.264651 (+ 1857us) server_negotiation.cc:407] Connection header received
0810 11:50:09.266774 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.266780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.266885 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.267012 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.268665 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.269394 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.271721 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.272142 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.272283 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.276095 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.276105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.276113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.276175 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.280157 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.280979 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.280987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.280994 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.281156 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.281719 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.281726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.281732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.282182 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.282349 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.283115 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.283379 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":2323,"threads_started":1}
I20260810 11:50:09.284610 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.261486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44319 (local address 127.0.0.1:43262)
0810 11:50:09.264465 (+ 2979us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.264514 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.266456 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.267277 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.267286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.267862 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.268440 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.268457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.269645 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.269649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.271570 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.271590 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.272377 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.272384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.272625 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.273332 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.273354 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.275825 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.280342 (+ 4517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.280349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.280363 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.280828 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.281305 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.281309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.281312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.281504 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.282327 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.282334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.282703 (+ 369us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.282949 (+ 246us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.283391 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2730,"thread_start_us":2483,"threads_started":1}
I20260810 11:50:09.286620 13423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.93.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.93.188@KRBTEST.COM" created.
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:09.302433 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.287394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53592)
0810 11:50:09.287598 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.287619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.287733 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.288963 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.288967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.289507 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.289710 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.289721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.291308 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.291317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.292270 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.292283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.293164 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.293170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.293430 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.294147 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.294165 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.296424 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.299891 (+ 3467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.299899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.299903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.300267 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.300661 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.300665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.300668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.300790 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.301395 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.301399 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.301542 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.301749 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.302027 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 11:50:09.302522 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.287798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.20.93.189:43705)
0810 11:50:09.288505 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.288511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.288547 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:50:09.288596 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.288602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.288800 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.288975 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.289912 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.291165 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.292427 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.293010 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.293156 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.296661 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.296674 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.296690 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.296719 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.299753 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.300398 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.300413 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.300415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.300525 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.300900 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.300907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.300910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.301257 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.301478 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.301921 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.302140 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":543,"thread_start_us":395,"threads_started":1}
I20260810 11:50:09.304949 13423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:09.306169 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.305696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47908)
0810 11:50:09.305870 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.306009 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:50:09.306476 13371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.308220 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.307816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47914)
0810 11:50:09.308016 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.308105 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 11:50:09.308445 13423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
Entry for principal kudu/127.20.93.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab.
I20260810 11:50:09.333487 20854 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFs48JS/build/asan/bin/kudu
/tmp/dist-test-taskFs48JS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:35181
--webserver_interface=127.20.93.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.93.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:50:09.357834 13423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 1
I20260810 11:50:09.359215 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.358702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47922)
0810 11:50:09.358920 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.359035 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 11:50:09.360330 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.359950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47930)
0810 11:50:09.360132 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.360213 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:50:09.360536 13423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.402619 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } attempt: 5
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:09.418021 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.403467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53604)
0810 11:50:09.403723 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.403748 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.403889 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.404293 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.404298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.404825 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.405082 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.405095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.406192 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.406198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.407549 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.407566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.408389 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.408396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.408612 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.409283 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.409308 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.411815 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.415391 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.415397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.415402 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.415826 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.416250 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.416255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.416259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.416405 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.416970 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.416975 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.417130 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.417282 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.417665 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 11:50:09.418021 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.403600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.20.93.189:43705)
0810 11:50:09.403790 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.403794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.403815 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:50:09.404052 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.404056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.404125 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.404219 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.405274 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.406048 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.407735 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.408222 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.408328 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.412045 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.412051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.412054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.412090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.415216 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.415965 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.415972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.415982 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.416126 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.416521 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.416527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.416530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.416835 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.416998 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.417398 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.417667 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:50:09.419112 13340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:09.420451 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.420030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47936)
0810 11:50:09.420230 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.420310 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 11:50:09.421479 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.421141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47938)
0810 11:50:09.421290 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.421380 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 11:50:09.421730 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.428762 13423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 2
I20260810 11:50:09.429906 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.429404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47946)
0810 11:50:09.429627 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.429733 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 11:50:09.431121 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.430709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47962)
0810 11:50:09.430908 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.430983 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 11:50:09.431331 13423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.472973 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 1
I20260810 11:50:09.474381 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.473817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47978)
0810 11:50:09.474072 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.474193 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 11:50:09.475523 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.475151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47982)
0810 11:50:09.475332 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.475410 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:50:09.475800 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.522576 13423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 3
I20260810 11:50:09.524071 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.523446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:47996)
0810 11:50:09.523693 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.523822 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 11:50:09.525362 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.524821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48008)
0810 11:50:09.525094 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.525208 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260810 11:50:09.525655 13423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.544085 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 2
I20260810 11:50:09.545432 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.544885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48024)
0810 11:50:09.545149 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.545251 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 11:50:09.546793 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.546386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48028)
0810 11:50:09.546546 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.546661 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:50:09.547191 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.653468 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 3
I20260810 11:50:09.654870 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.654331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48042)
0810 11:50:09.654582 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.654684 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 11:50:09.656111 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.655641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48050)
0810 11:50:09.655889 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.655974 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260810 11:50:09.656394 13340 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
I20260810 11:50:09.668903 13423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 4
I20260810 11:50:09.670274 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.669754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48056)
0810 11:50:09.670003 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.670115 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:50:09.671334 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.671013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48062)
0810 11:50:09.671155 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.671230 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 11:50:09.671594 13423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.93.188:35181: Network error: Client connection negotiation failed: client connection to 127.20.93.188:35181: connect: Connection refused (error 111)
W20260810 11:50:09.712673 13433 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:50:09.713063 13433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:50:09.713187 13433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:50:09.713323 13433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:50:09.721192 13433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:50:09.721299 13433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:50:09.721390 13433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:50:09.721467 13433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:50:09.721624 13433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:50:09.732110 13433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.93.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.93.190:44319,127.20.93.189:43705,127.20.93.188:35181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/krb5kdc/kudu_127.20.93.188.keytab
--principal=kudu/127.20.93.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.93.188:35181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.93.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:50:09.733584 13433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:50:09.735329 13433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:50:09.748131 13439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:09.748293 13442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:50:09.748335 13440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:50:09.749245 13433 server_base.cc:1034] running on GCE node
I20260810 11:50:09.750270 13433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:50:09.751947 13433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:50:09.753211 13433 hybrid_clock.cc:648] HybridClock initialized: now 1786362609753123 us; error 65 us; skew 500 ppm
I20260810 11:50:09.753633 13433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:50:09.757051 13433 init.cc:377] Logged in from keytab as kudu/127.20.93.188@KRBTEST.COM (short username kudu)
I20260810 11:50:09.758772 13433 webserver.cc:460] Webserver started at http://127.20.93.188:39819/ using document root <none> and password file <none>
I20260810 11:50:09.759558 13433 fs_manager.cc:359] Metadata directory not provided
I20260810 11:50:09.759713 13433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:50:09.760069 13433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:50:09.763315 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data/instance:
uuid: "f42b9e547ae240dd938ab4126b014881"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "301ef4be546e18845c809448e097c39c"
server_key_iv: "eda6caa3f415e36edf00d79916c4db93"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.764184 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal/instance:
uuid: "f42b9e547ae240dd938ab4126b014881"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "301ef4be546e18845c809448e097c39c"
server_key_iv: "eda6caa3f415e36edf00d79916c4db93"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.770131 13433 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:50:09.774133 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:50:09.775906 13433 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:50:09.776144 13433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data,/tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal
uuid: "f42b9e547ae240dd938ab4126b014881"
format_stamp: "Formatted at 2026-08-10 11:50:09 on dist-test-slave-njxd"
server_key: "301ef4be546e18845c809448e097c39c"
server_key_iv: "eda6caa3f415e36edf00d79916c4db93"
server_key_version: "encryptionkey@0"
I20260810 11:50:09.776407 13433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:50:09.796458 13433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:50:09.801906 13433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:50:09.802253 13433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:50:09.822294 13433 rpc_server.cc:307] RPC server started. Bound to: 127.20.93.188:35181
I20260810 11:50:09.822302 13506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.93.188:35181 every 8 connection(s)
I20260810 11:50:09.823735 13433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/data/info.pb
I20260810 11:50:09.827526 13507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:50:09.832005 20854 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFs48JS/build/asan/bin/kudu as pid 13433
I20260810 11:50:09.832206 20854 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFs48JS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786362228785499-20854-0/minicluster-data/master-2/wal/instance
I20260810 11:50:09.832779 20854 external_mini_cluster.cc:1428] Setting key 1a34de947e4432ae76aabe62cabde9b6
I20260810 11:50:09.833668 13340 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 4
I20260810 11:50:09.836385 13507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.190@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:09.859099 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.834391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48088)
0810 11:50:09.834796 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.834823 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.834975 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.839848 (+ 4873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.839855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.840419 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.840665 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.840684 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.844122 (+ 3438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.844130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.845810 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.845827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.847116 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.847123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.847343 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.848139 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.848163 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.850856 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.855511 (+ 4655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.855517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.855522 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.856172 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.856649 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.856654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.856659 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.856801 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.858003 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.858007 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.858187 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.858386 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.858717 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":1}
I20260810 11:50:09.860735 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.836223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.20.93.188:35181)
0810 11:50:09.836980 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.836986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.837011 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:50:09.838839 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.838855 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.839408 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.840598 (+ 1190us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.840902 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.843937 (+ 3035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.846027 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.846914 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.847763 (+ 849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.851132 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.851148 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.851170 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.851221 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.855360 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.856326 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.856336 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.856353 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.856514 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.856957 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.856962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.856966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.857831 (+ 865us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.858043 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.858668 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.859159 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":599,"spinlock_wait_cycles":30464,"thread_start_us":416,"threads_started":1}
I20260810 11:50:09.860806 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.834736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.20.93.188:35181)
0810 11:50:09.836149 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.836167 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.836224 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:50:09.838821 (+ 2597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.838848 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.839406 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.839828 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.842789 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.843939 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.845989 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.846910 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.847149 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.852450 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.852458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.852461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.852498 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.856349 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.857404 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.857409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.857411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.857531 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.858045 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.858053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.858058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.858426 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.859967 (+ 1541us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.860075 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.860313 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":261,"server-negotiator.queue_time_us":1086,"spinlock_wait_cycles":4992,"thread_start_us":831,"threads_started":1}
I20260810 11:50:09.862370 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.841410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.190:44319 (local address 127.0.0.1:43272)
0810 11:50:09.842457 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.842506 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.842648 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.844335 (+ 1687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.844343 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.845095 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.845351 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.845360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.846859 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.846864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.848606 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.848620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.849994 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.850002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.850216 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.850989 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.851032 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.854539 (+ 3507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.858775 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.858788 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.858795 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.859403 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.859893 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.859902 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.859908 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.860075 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.860751 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.860765 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.861190 (+ 425us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.861402 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.861775 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":632,"threads_started":1}
I20260810 11:50:09.862418 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.841193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.20.93.190:44319)
0810 11:50:09.843889 (+ 2696us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.843898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.843919 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:50:09.843978 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.843982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.844086 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.844254 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.845531 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.846642 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.848793 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.849694 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.852229 (+ 2535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.854802 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.854811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.854818 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.854866 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.858623 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.859580 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.859589 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.859595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.859742 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.860207 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.860212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.860218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.860561 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.860703 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.861782 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.862026 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2570,"thread_start_us":2391,"threads_started":1}
I20260810 11:50:09.864153 13507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:09.868763 13340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: Bootstrap starting.
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:09.873267 13340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:09.874814 13340 log.cc:824] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.190@KRBTEST.COM
I20260810 11:50:09.878482 13340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: No bootstrap required, opened a new log
I20260810 11:50:09.880271 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.864816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.189:43705 (local address 127.0.0.1:53610)
0810 11:50:09.865166 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.865183 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.865278 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.865721 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.865729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.866235 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.866442 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.866454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.867851 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.867857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.868802 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.868815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.869939 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.869945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.870143 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.870887 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.870905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.873202 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.877119 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.877125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.877129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.877742 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.878190 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.878196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.878199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.878354 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.879183 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.879188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.879350 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.879505 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.879852 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260810 11:50:09.880338 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.865162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.20.93.189:43705)
0810 11:50:09.865338 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.865343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.865362 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:50:09.865420 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.865427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.865500 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.865637 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.866629 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.867670 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.869015 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.869506 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.869665 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.873426 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.873432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.873436 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.873471 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.876953 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.877889 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.877898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.877906 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.878053 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.878469 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.878474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.878478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.879042 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.879204 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.879600 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.879851 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:50:09.882026 13507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:50:09.884835 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.864816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.20.93.190:44319)
0810 11:50:09.864977 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.864983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.865002 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:50:09.865057 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.865062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.865186 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.866932 (+ 1746us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.867045 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.867828 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.869667 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.870315 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.870465 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.876715 (+ 6250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.876726 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.876732 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.876771 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.880416 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.881466 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.881474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.881483 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.881699 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.882288 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.882294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.882297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.882718 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.883053 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.884190 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.884460 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 11:50:09.889758 13340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.890229 13340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e00779a47e34a6ebac7685441be3074, State: Initialized, Role: FOLLOWER
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.188@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:09.890775 13340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.891826 13516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:09.892227 13516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:09.893321 13340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.93.189@KRBTEST.COM
I20260810 11:50:09.898952 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.883186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.20.93.188:35181)
0810 11:50:09.883387 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.883394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.883435 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:50:09.883566 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.883572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.883646 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.883751 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.884702 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.885891 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.887376 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.887930 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.888037 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.892081 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.892087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.892092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.892135 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.895599 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.896507 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.896516 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.896520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.896690 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.897150 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.897157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.897163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.897617 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.897743 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.898323 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.898564 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:50:09.899897 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.882834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48090)
0810 11:50:09.883365 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.883384 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.883469 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.883801 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.883806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.884321 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.884525 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.884537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.886017 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.886025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.887207 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.887226 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.888272 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.888277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.888474 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.889168 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.889193 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.891885 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.895768 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.895773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.895778 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.896372 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.896841 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.896847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.896853 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.897017 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.897835 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.897841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.898025 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.899166 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.899491 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418}
W20260810 11:50:09.905122 13530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:50:09.905285 13530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:50:09.906378 13507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: Bootstrap starting.
I20260810 11:50:09.910231 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.887456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.20.93.189:43705)
0810 11:50:09.887649 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.887664 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.887683 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:50:09.887744 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.887750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.887831 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.887946 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.889747 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.890829 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.892663 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.893669 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.894104 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.901336 (+ 7232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.901344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.901348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.901399 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.906112 (+ 4713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.907345 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.907350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.907354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.907535 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.908133 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.908139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.908142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.908572 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.908902 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.909522 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.909889 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:50:09.910933 13507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:09.912451 13507 log.cc:824] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: Log is configured to *not* fsync() on all Append() calls
I20260810 11:50:09.916837 13507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: No bootstrap required, opened a new log
I20260810 11:50:09.916982 20854 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:50:09.906917 13266 master_runner.cc:186] Error getting master registration for 127.20.93.189:43705: OK, instance_id { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" instance_seqno: 1786362609239841 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:50:09.918562 13266 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:50:09.920388 13313 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5e00779a47e34a6ebac7685441be3074"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286
I20260810 11:50:09.920855 13313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 11:50:09.921068 13313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 0 FOLLOWER]: Advancing to term 1
W20260810 11:50:09.924126 13291 tablet.cc:2369] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:50:09.924978 13313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.927721 13313 leader_election.cc:288] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [CANDIDATE]: Term 1 election: Requested vote from peers f34313b57c4d4c36851a5012a95d77b0 (127.20.93.189:43705), f42b9e547ae240dd938ab4126b014881 (127.20.93.188:35181)
I20260810 11:50:09.927533 13507 raft_consensus.cc:348] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.928002 13507 raft_consensus.cc:729] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42b9e547ae240dd938ab4126b014881, State: Initialized, Role: FOLLOWER
I20260810 11:50:09.928637 13507 consensus_queue.cc:260] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.929952 13533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:09.930691 13533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:09.931342 13507 sys_catalog.cc:564] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:09.931391 13479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e00779a47e34a6ebac7685441be3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f42b9e547ae240dd938ab4126b014881"
I20260810 11:50:09.931854 13479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:09.936808 13479 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e00779a47e34a6ebac7685441be3074 in term 1.
I20260810 11:50:09.938175 13287 leader_election.cc:302] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e00779a47e34a6ebac7685441be3074, f42b9e547ae240dd938ab4126b014881; no voters:
I20260810 11:50:09.938912 13516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:50:09.940271 13516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [term 1 LEADER]: Becoming Leader. State: Replica: 5e00779a47e34a6ebac7685441be3074, State: Running, Role: LEADER
I20260810 11:50:09.941030 13516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:09.955032 13540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e00779a47e34a6ebac7685441be3074. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:09.955638 13540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:09.968750 13552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:50:09.970803 13423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } attempt: 5
I20260810 11:50:09.975572 13552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 11:50:09 dist-test-slave-njxd krb5kdc[13252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786362609, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.93.189@KRBTEST.COM for kudu/127.20.93.188@KRBTEST.COM
I20260810 11:50:09.993263 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.971603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.20.93.188:35181)
0810 11:50:09.971810 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.971819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.971845 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:50:09.971930 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.971935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.972021 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.972141 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.973391 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.974503 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.976327 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.977016 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.977158 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.983712 (+ 6554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.983720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.983725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.983778 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:09.987797 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.989181 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.989196 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.989199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.989362 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:09.990083 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:09.990088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:09.990092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:09.990487 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:09.990627 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:09.991565 (+ 938us) server_negotiation.cc:299] Negotiation successful
0810 11:50:09.992639 (+ 1074us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 11:50:09.995121 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.971476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.93.188:35181 (local address 127.0.0.1:48112)
0810 11:50:09.971646 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:50:09.971668 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:50:09.971793 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:50:09.972383 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:50:09.972391 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:50:09.972914 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:50:09.973164 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.973182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.974696 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.974702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.976154 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:50:09.976173 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.977205 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.977213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:50:09.977499 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.978582 (+ 1083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:50:09.978628 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:50:09.983371 (+ 4743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:50:09.988049 (+ 4678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.988056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.988061 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.988963 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.989620 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:50:09.989625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:50:09.989629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:50:09.989885 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:50:09.990802 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:50:09.990810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:50:09.991065 (+ 255us) client_negotiation.cc:769] Sending connection context
0810 11:50:09.994005 (+ 2940us) client_negotiation.cc:240] Negotiation successful
0810 11:50:09.994536 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 11:50:09.996400 13557 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:50:10.002246 13423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: Bootstrap starting.
I20260810 11:50:09.998865 13479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [term 1 FOLLOWER]: Refusing update from remote peer 5e00779a47e34a6ebac7685441be3074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:50:10.004925 13540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:50:10.009430 13423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: Neither blocks nor log segments found. Creating new log.
I20260810 11:50:10.011811 13423 log.cc:824] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:50:10.026176 13423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: No bootstrap required, opened a new log
I20260810 11:50:10.031394 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.951717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.20.93.188:35181)
0810 11:50:09.951983 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.951992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.960918 (+ 8926us) server_negotiation.cc:407] Connection header received
0810 11:50:09.962572 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.962580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.962685 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.962773 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.972185 (+ 9412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.973382 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.980176 (+ 6794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.980864 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.980997 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:09.999119 (+ 18122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:09.999126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:09.999130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:09.999171 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:10.003741 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.010914 (+ 7173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.010922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.010929 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.011355 (+ 426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.012004 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.012009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.012013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.012493 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:10.012770 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:10.030454 (+ 17684us) server_negotiation.cc:299] Negotiation successful
0810 11:50:10.030884 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:50:10.043633 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.961643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.20.93.189:43705)
0810 11:50:09.961909 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.961919 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.968026 (+ 6107us) server_negotiation.cc:407] Connection header received
0810 11:50:09.969088 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.969094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.969183 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.969280 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.976435 (+ 7155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.977659 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.989904 (+ 12245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.990589 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.990688 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:10.001356 (+ 10668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:10.001364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:10.001372 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:10.001414 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:10.010864 (+ 9450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.033430 (+ 22566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.033446 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.033453 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.033749 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.039300 (+ 5551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.039307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.039313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.039918 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:10.040251 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:10.042678 (+ 2427us) server_negotiation.cc:299] Negotiation successful
0810 11:50:10.043077 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:50:10.045962 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:09.951081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.20.93.190:44319)
0810 11:50:09.951308 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:09.951315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:09.968018 (+ 16703us) server_negotiation.cc:407] Connection header received
0810 11:50:09.968734 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:09.968741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:09.968845 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:09.968936 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:09.977948 (+ 9012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.979235 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.991697 (+ 12462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:09.992444 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:09.992555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:10.023614 (+ 31059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:50:10.023626 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:50:10.023635 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:50:10.023701 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:50:10.028961 (+ 5260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.038384 (+ 9423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.038393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.038399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.038662 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:50:10.040906 (+ 2244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:50:10.040915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:50:10.040920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:50:10.041470 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:50:10.041906 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:10.045006 (+ 3100us) server_negotiation.cc:299] Negotiation successful
0810 11:50:10.045365 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 11:50:10.011086 13433 master_runner.cc:186] Error getting master registration for 127.20.93.189:43705: OK, instance_id { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" instance_seqno: 1786362609239841 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:50:10.050588 13433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:50:10.050025 13423 raft_consensus.cc:348] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
I20260810 11:50:10.050812 13423 raft_consensus.cc:729] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34313b57c4d4c36851a5012a95d77b0, State: Initialized, Role: FOLLOWER
I20260810 11:50:10.051692 13423 consensus_queue.cc:260] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } }
W20260810 11:50:10.054617 13457 tablet.cc:2369] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:50:10.061352 13423 sys_catalog.cc:564] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:50:10.061420 13395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e00779a47e34a6ebac7685441be3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f34313b57c4d4c36851a5012a95d77b0"
I20260810 11:50:10.062101 13395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:50:10.062626 13533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e00779a47e34a6ebac7685441be3074. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.062953 13533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:10.059252 13561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.063130 13561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:10.065954 13395 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e00779a47e34a6ebac7685441be3074 in term 1.
I20260810 11:50:10.066902 13556 mvcc.cc:205] Tried to move back new op lower bound from 7316941250524184576 to 7316941250331185152. Current Snapshot: MvccSnapshot[applied={T|T < 7316941250524184576}]
I20260810 11:50:10.067659 13394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [term 1 FOLLOWER]: Refusing update from remote peer 5e00779a47e34a6ebac7685441be3074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:50:10.068585 13540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:50:10.068697 13533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.068922 13533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:10.080829 13540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.081224 13540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:10.083673 13552 catalog_manager.cc:1347] Generated new cluster ID: e1690bebd6d942238e7f44fe9392a026
I20260810 11:50:10.083822 13552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:50:10.096364 13573 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.096762 13573 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:10.097369 13561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e00779a47e34a6ebac7685441be3074. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.097657 13561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:50:10.112567 13516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e00779a47e34a6ebac7685441be3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e00779a47e34a6ebac7685441be3074" member_type: VOTER last_known_addr { host: "127.20.93.190" port: 44319 } } peers { permanent_uuid: "f34313b57c4d4c36851a5012a95d77b0" member_type: VOTER last_known_addr { host: "127.20.93.189" port: 43705 } } peers { permanent_uuid: "f42b9e547ae240dd938ab4126b014881" member_type: VOTER last_known_addr { host: "127.20.93.188" port: 35181 } } }
I20260810 11:50:10.113020 13516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074 [sys.catalog]: This master's current role is: LEADER
I20260810 11:50:10.116752 13552 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:50:10.118762 13552 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:50:10.122407 13584 catalog_manager.cc:1259] Loaded cluster ID: e1690bebd6d942238e7f44fe9392a026
I20260810 11:50:10.122514 13584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: loading cluster ID for follower catalog manager: success
I20260810 11:50:10.126416 13584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: acquiring CA information for follower catalog manager: success
W20260810 11:50:10.127797 13584 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260810 11:50:10.141758 13552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e00779a47e34a6ebac7685441be3074: Generated new TSK 0
I20260810 11:50:10.143446 13552 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:50:10.262064 13513 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.253138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.20.93.190:44319)
0810 11:50:10.253370 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.253379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.258800 (+ 5421us) server_negotiation.cc:407] Connection header received
0810 11:50:10.260329 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.260337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.260436 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.260516 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.261815 (+ 1299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43318: BlockingRecv error: recv got EOF from 127.0.0.1:43318 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 11:50:10.267846 13510 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.247124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.20.93.188:35181)
0810 11:50:10.247328 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.247337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.258305 (+ 10968us) server_negotiation.cc:407] Connection header received
0810 11:50:10.265963 (+ 7658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.265973 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.266114 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.266206 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.267544 (+ 1338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48122: BlockingRecv error: recv got EOF from 127.0.0.1:48122 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 11:50:10.269073 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.259622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.20.93.189:43705)
0810 11:50:10.259857 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.259863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.263353 (+ 3490us) server_negotiation.cc:407] Connection header received
0810 11:50:10.267545 (+ 4192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.267551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.267659 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.267755 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.268839 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53632: BlockingRecv error: recv got EOF from 127.0.0.1:53632 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 11:50:10.296140 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.293460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.20.93.189:43705)
0810 11:50:10.293789 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.293799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.294443 (+ 644us) server_negotiation.cc:407] Connection header received
0810 11:50:10.294745 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.294751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.294875 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.295009 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.295835 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53634: BlockingRecv error: recv got EOF from 127.0.0.1:53634 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
W20260810 11:50:10.302591 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.300346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.20.93.189:43705)
0810 11:50:10.300605 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.300614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.301137 (+ 523us) server_negotiation.cc:407] Connection header received
0810 11:50:10.301356 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.301361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.301447 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.301565 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.302368 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53650: BlockingRecv error: recv got EOF from 127.0.0.1:53650 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 11:50:10.310312 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.307948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.20.93.189:43705)
0810 11:50:10.308184 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.308194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.308707 (+ 513us) server_negotiation.cc:407] Connection header received
0810 11:50:10.308968 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.308993 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.309113 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.309211 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.310022 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53652: BlockingRecv error: recv got EOF from 127.0.0.1:53652 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 11:50:10.319796 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.317427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.20.93.189:43705)
0810 11:50:10.317768 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.317778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.318336 (+ 558us) server_negotiation.cc:407] Connection header received
0810 11:50:10.318568 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.318573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.318660 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.318775 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.319551 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53662: BlockingRecv error: recv got EOF from 127.0.0.1:53662 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260810 11:50:10.333487 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.331083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.20.93.189:43705)
0810 11:50:10.331420 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.331430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.331985 (+ 555us) server_negotiation.cc:407] Connection header received
0810 11:50:10.332220 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.332228 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.332337 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.332451 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.333242 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53666: BlockingRecv error: recv got EOF from 127.0.0.1:53666 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
W20260810 11:50:10.355386 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.352949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.20.93.189:43705)
0810 11:50:10.353210 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.353220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.353853 (+ 633us) server_negotiation.cc:407] Connection header received
0810 11:50:10.354099 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.354104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.354198 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.354316 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.355122 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53674: BlockingRecv error: recv got EOF from 127.0.0.1:53674 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 11:50:10.393384 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.390930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.20.93.189:43705)
0810 11:50:10.391180 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.391190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.391795 (+ 605us) server_negotiation.cc:407] Connection header received
0810 11:50:10.392032 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.392041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.392134 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.392248 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.393086 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53690: BlockingRecv error: recv got EOF from 127.0.0.1:53690 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:50:10.463279 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.460833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.20.93.189:43705)
0810 11:50:10.461083 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.461093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.461646 (+ 553us) server_negotiation.cc:407] Connection header received
0810 11:50:10.461960 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.461969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.462051 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.462187 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.463033 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53706: BlockingRecv error: recv got EOF from 127.0.0.1:53706 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 11:50:10.597648 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.595077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.20.93.189:43705)
0810 11:50:10.595368 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.595375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.595743 (+ 368us) server_negotiation.cc:407] Connection header received
0810 11:50:10.595961 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.595965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.596041 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.596151 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.597291 (+ 1140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53712: BlockingRecv error: recv got EOF from 127.0.0.1:53712 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 11:50:10.861552 13428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:50:10.859371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.20.93.189:43705)
0810 11:50:10.859645 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:10.859653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:10.859884 (+ 231us) server_negotiation.cc:407] Connection header received
0810 11:50:10.860151 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:10.860156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:10.860263 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:10.860387 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:50:10.861306 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53728: BlockingRecv error: recv got EOF from 127.0.0.1:53728 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 11:50:11.001272 13557 catalog_manager.cc:1259] Loaded cluster ID: e1690bebd6d942238e7f44fe9392a026
I20260810 11:50:11.001461 13557 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: loading cluster ID for follower catalog manager: success
I20260810 11:50:11.004318 13557 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: acquiring CA information for follower catalog manager: success
I20260810 11:50:11.006703 13557 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f42b9e547ae240dd938ab4126b014881: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:50:11.130865 13584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f34313b57c4d4c36851a5012a95d77b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:50:11.386094 13664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:11.379501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.20.93.189:43705)
0810 11:50:11.380442 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:11.380447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:11.380467 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:50:11.380543 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:11.380547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:11.380706 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:11.380827 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:11.381376 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.382258 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.383623 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.384048 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.384148 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:11.384641 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:11.385205 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:11.385335 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:11.385552 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 11:50:11.385705 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":616,"threads_started":1}
I20260810 11:50:11.401108 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:11.394059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.20.93.188:35181)
0810 11:50:11.394723 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:11.394732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:11.394782 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:50:11.395090 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:11.395094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:11.395256 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:11.395447 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:11.395999 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.396992 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.398561 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.398994 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.399171 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:11.399776 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:11.400386 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:11.400583 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:11.400642 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 11:50:11.400733 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":334,"threads_started":1}
I20260810 11:50:11.415892 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:11.407753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.20.93.188:35181)
0810 11:50:11.408151 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:11.408155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:11.408950 (+ 795us) server_negotiation.cc:407] Connection header received
0810 11:50:11.409287 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:11.409293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:11.409411 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:11.409484 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:11.410375 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.411309 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.413676 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.414292 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.414442 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:11.414959 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:11.415065 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:11.415166 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:11.415328 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 11:50:11.415413 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260810 11:50:11.416674 13664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:11.407943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.20.93.189:43705)
0810 11:50:11.408299 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:11.408306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:11.409653 (+ 1347us) server_negotiation.cc:407] Connection header received
0810 11:50:11.409930 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:11.409938 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:11.410034 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:11.410087 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:11.410936 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.411975 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.414191 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.414827 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.414925 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:11.415619 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:11.415771 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:11.415856 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:11.416141 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 11:50:11.416229 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260810 11:50:11.416702 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:50:11.407266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.20.93.190:44319)
0810 11:50:11.409111 (+ 1845us) server_negotiation.cc:206] Beginning negotiation
0810 11:50:11.409119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:50:11.409148 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:50:11.409291 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:50:11.409297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:50:11.409548 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:50:11.409776 (+ 228us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:50:11.410422 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.411342 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.413357 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:50:11.413929 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:50:11.414076 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:50:11.414703 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:50:11.415235 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:50:11.415808 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0810 11:50:11.415865 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 11:50:11.416230 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1660,"thread_start_us":1549,"threads_started":1}
I20260810 11:50:11.422096 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 13266
I20260810 11:50:11.423434 13335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:11.541693 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 13266
I20260810 11:50:11.569144 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 13347
I20260810 11:50:11.570358 13418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:11.682260 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 13347
I20260810 11:50:11.707252 20854 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFs48JS/build/asan/bin/kudu pid 13433
I20260810 11:50:11.708386 13502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:50:11.818215 20854 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFs48JS/build/asan/bin/kudu with pid 13433
2026-08-10T11:50:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5159 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9916 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (382997 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST